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Executive Summary 


This report presents the results of an exploratory study 
of suspension, discipline, and climate in K-5 and K-8 
schools in the School District of Philadelphia (SDP). 

The study was conducted between January 2016 and 
October 2017 by the Consortium for Policy Research in 
Education (CPRE) at the University of Pennsylvania, in 
close partnership with SDP and with research support 
from Research for Action, a Philadelphia-based 
research organization. The study was funded by a 2015 
Comprehensive School Safety Initiative grant from 

the National Institute of Justice, a division of the U.S. 
Department of Justice. 


Chapter 1 of the report describes the context and 
need for this study. The research was conducted in 
response to requests from SDP leadership for more 
information about how schools are managing student 
discipline in the wake of recent district-wide policy and 
programmatic changes. These changes were designed 
to improve school climate and reduce the use of 
out-of-school suspension (OSS), and they include the 
rollout of Positive Behavior Interventions and Supports 
(PBIS) in a growing number of SDP schools. The findings 
presented here are informed by data collected in 166 
schools, representing 96.5% of the K-5 and K-8 schools 
managed by SDP, in order to answer the following 
research questions: 


1. What disciplinary practices are used by SDP 
schools serving students in kindergarten through 
eighth grade? 


2. What factors support or hinder schools’ alignment 
with the district’s climate and suspension- 
reduction goals? 


3. What patterns are evident in SDP K-5 and K-8 
schools’ approaches to discipline and climate? 


4. Areschools’ approaches to discipline and climate 
related to suspension and academic outcomes? 


5. How are different approaches to discipline and 
climate manifested in individual schools? 


In Chapter 2 we present the research methods and 
findings related to research questions 1 and 2. These 
questions were addressed via mixed-methods inquiry 
that included both surveys and in-depth qualitative 
research. The qualitative research was conducted 
between January 2016 and June 2017, and included a 
series of focus groups and one-on-one interviews with 
teachers, principals, and other school staff. Data were 
also collected through online surveys administered to 
teachers and school administrators in all SDP K-5 and 
K-8 schools during the spring of 2016. 


Via these methods, we identified four key findings 
pertaining to the disciplinary practices SDP schools 
use, and the factors that impede and/or foster 

their shifts away from OSS and toward a focus on 
improving climate. First, we found that most schools 
are seeking alternatives to suspension in response to 
student misbehavior. The extent to which this is an 
explicit priority varies across schools generally, and 
between PBIS and non-PBIS schools. Second, we found 
that a subset of schools is taking steps to proactively 
address student behavior through programming that 
encourages positive choices and engagement. This 
subgroup includes a combination of PBIS and non- 
PBIS schools. Third, we found that administrators and 
teachers identify resource shortages in the areas of 
staffing, soace, and supportive services as the biggest 
impediment to reducing the use of suspension. And 
finally, we conclude that administrators and teachers 
are often not aligned about how discipline should be 
managed and whether and when suspension should 
be used. This is the case in both PBIS and non-PBIS 
schools. 


Chapter 3 details the statistical methods we used to 
address research questions 3 and 4, and presents the 
findings of these analyses. In response to question 3, 
we used latent class analysis to identify a typology of 
SDP K-5 and K-8 schools’ approaches to discipline and 
climate. Data for these analyses were obtained from 
teacher and principal surveys. Ultimately, we identified 
three profiles of schools based on respondents’ 
characterizations of climate and disciplinary practices. 
The three profiles are: 


Profile 1. Reactive and autonomous. These schools rely 
on punitive and exclusionary disciplinary responses 
to maintain order. OSS is regarded as an important 
means of keeping control. Teachers perceive that 
they must fend for themselves with regard to discipline. 


Profile 2. Under-resourced and non-cohesive. Schools 
in this profile experience staffing and resource 
shortages most profoundly, and are inconsistent in 
their use of both punitive and non-punitive practices. 
Staff report low morale. Teachers report little 
collaboration around discipline, and feel blamed by 
administrators for their students’ misbehavior. 


Profile 3. Collaborative and relational. These schools 
are characterized by collaborative approaches 

to discipline and the use of non-punitive practices. 
Morale is high; teachers feel supported. OSS is not 
viewed as an effective response to misbehavior. 


Compared with schools in Profiles 1 and 2, those in 
Profile 3 were more likely to be located in communities 
with lower poverty, higher educational attainment, 
fewer households without English spoken in the home, 
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higher percentage of White non-Hispanic residents, 
and fewer minors living with one parent. In addition, 
schools implementing PBIS with SDP support were 
dispersed among all three profiles and were no more 
likely to be in Profile 3 than in the other two profiles. This 
suggests poor penetration of PBIS practices in many 
implementing schools. 


In response to research question 4, we used event 
history analysis and 2015-2016 SDP data to examine the 
extent to which profile membership predicts student 
outcomes, including OSS and academic achievement. 
Controlling for student demographics and other 
factors, we observed that students in Profile 3 schools 
had a lower likelihood of out-of-school suspension, 

and scored significantly higher on state assessments in 
English, Math, and Science. 


In response to research question 5, four case studies 
of individual schools are presented in Chapter 4. These 
schools were drawn from Profiles 1 and 3, occupy 
different parts of the city, and serve different student 
populations. Two of the schools are implementing 

PBIS with SDP support. Our analysis of these four cases 
together underscores the key findings of Chapter 2 
about how schools address climate and discipline and 
the obstacles they face in doing so. In addition, this 
analysis highlights how individual schools’ contexts 
shape their climate and discipline successes and 
challenges. 


Chapter 5 highlights the implications of the study’s 
overall findings. We conclude that climate- 
improvement efforts like PBIS hold great promise for 
improving student outcomes as well as the experience 
of school for staff, students, and families in SDP. In 
order to realize this potential, we recommend that 
SDP embrace a PBIS implementation approach that 

is tailored to the challenges of its context — more 
specifically, one that emphasizes differentiated training 
and intensive support for all adults, and pairs climate 
efforts with a focus on trauma-informed care. 
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Chapter One. 
National Issues, Local Realities 


This report presents the results of an 18-month 
exploratory study of disciplinary practices and 
climate in K-5 and K-8 schools in the School District of 
Philadelphia (SDP). The study was conducted by the 
Consortium for Policy Research in Education (CPRE) 
at the University of Pennsylvania, in close partnership 
with the SDP and with support from Research for 
Action, a Philadelphia-based research organization. It 
was funded by a 2015 Comprehensive School Safety 
Initiative grant from the National Institute of Justice, a 
division of the U.S. Department of Justice. 


The study was conducted in response to requests 

from SDP for more information about how schools are 
managing student discipline in the wake of recent 
district-wide policy and programmatic changes 
designed to improve school climate and reduce the 
use of out-of-school suspension (OSS). The research 
focused specifically on disciplinary practices in SDP 
elementary and middle schools. Because these schools 
were shown in recent research to suspend students at 
higher rates than high schools in the district (Engelman 
& Wolford, 2014), the impact of SDP’s suspension- 
reduction efforts on the district’s K-5 and K-8 schools 
was of particular interest. 


In this report, we address both disciplinary approach— 
by which we mean the way a school prevents and/ 

or responds to student misbehavior—and school 
climate. School climate refers, more generally, to the 
environment a school’s students and staff experience, 
and encompasses disciplinary approach as well as 
norms, expectations, and relationships. As our research 
illustrates, these two issues are sometimes, but not 
always, addressed simultaneously. 


The findings presented here are informed by data 
collected in 166 schools, representing 96.5% of the K-5 
and K-8 schools managed by SDP. (A small number of 
schools were not included due to low survey response 
rates.) Because a central goal of the study was to 
inform SDP's policies and practices related to climate 
and discipline, it does not include charter schools. 


Out-of-school suspension: 
A national and local problem 


Serving some 130,000 students in 218 schools, SDP is 
the eighth largest public school district in the country 
and one of the most diverse. It is among the nation’s 
most financially and academically challenged school 
districts (Steinberg & Quinn, 2014; Cornman, 2013; Wills, 
Karakus, & Wolford, 2017): Nearly 90% of SDP students 
qualify for free lunch, most are historically underserved 
racial minorities! (Wills, Karakus, & Wolford, 2017), and 
two-thirds have experienced traumatic events like 
poverty, violence, familial instability, or abuse (Hardy, 
2014). District schools have been rocked by severe 
budget restrictions following the loss of millions in state 
funding; as recently as 2014-15, most K-5 and K-8 
schools in the district lacked full-time nurses, counselors, 
and in some cases even receptionists. Amid these 
challenges, SDP is an example of an urban district 

that is working to change disciplinary practices and 
improve climate in its schools. 


SDP’s focus on school climate has emerged over the 
past several years, largely in response to alarming 
revelations about the detrimental impacts of 
exclusionary disciplinary practices—OSS in particular. 
Although OSS has been a widely used disciplinary 


1 SDP reports that in 2016-17 its student body was 50% black/African American; 20% Hispanic, 14% white, 8% Asian, and 7% multi-race. 


9 Consortium for Policy Research in Education | RR 2017-4 


Discipline in Context: Suspension, Climate, and PBIS in the School District of Philadelphia 


intervention in elementary, middle, and high schools 
for decades, recent research demonstrates that 

the practice is not only ineffective as a means of 
improving behavior (Finn & Servoss, 2015; Fabelo et 
al., 2011; Skiba, Poloni-Saudinger, Gallini, Simmons, 
& Feggins-Azziz, 2006), but also overtly harmful to 
students. Multiple well-publicized studies have linked 
OSS with increased rates of academic failure, school 
dropout, misbehavior, criminal involvement, and 
incarceration (Anyon et al., 2016; Gregory, Clawson, 
Davis, & Gerewitz, 2016; Balfanz, Byrnes, & Fox, 2015; 
Marchbanks et al., 2015). Furthermore, analysis of 
national disciplinary data reveals disproportionate 
use of OSS with Black, Latino, and special education 
students, and English language learners (Losen & 
Martinez, 2013; Losen, Ee, Hodson, & Martinez, 2015; 


Office of Civil Rights, 2016; Skiba, Arredondo, & Rausch, 


2014). These national patterns of disproportionality 
are largely replicated in SDP’s own analyses of district 
schools’ suspension data (Wills, Karakus, & Wolford, 
2017). 


Growing numbers of state and local education 
agencies are responding to research on the negative 
impacts of OSS with policy changes designed to 

limit the use of exclusionary practices in schools. SDP 
has taken several steps in this direction, including 
penalizing schools that overuse OSS in the district's 
School Progress Report evaluation process, and 
prohibiting the suspension of kindergarten students 
altogether (School Reform Commission of the School 
District of Philadelphia, 2016). In addition, SDP 

revised its Code of Student Conduct (School Reform 
Commission of the School District of Philadelphia, 2013) 
in 2013 to raise the bar on OSS at all grade levels: 
Offenses once considered OSS-worthy—like uniform 
policy violations—now should result in less severe 
consequences like detentions or parent contacts. 


SDP administrators identify these policy changes as 
primary mechanisms for communicating the district’s 
priorities for climate and discipline to school leaders 
and staff. The key messages SDP leaders hope to 
convey, according to district-level administrators, 
are that OSS is not an effective way to change 
students’ behavior and that schools need to identify 
alternative disciplinary interventions. More broadly, 
by discouraging the use of exclusionary practices, the 
district hopes to help facilitate a “culture shift” in the 
ways SDP schools teach, manage, and respond to 
student behavior. 


The School District of Philadelphia’s 
investment in climate 


To achieve its goal of facilitating a shift in schools’ 
disciplinary approaches and climate, SDP has taken 
steps not only to reduce the use of OSS, but also to 
promote the use of non-punitive disciplinary practices 
more generally. For purposes of this study, non-punitive 
practices include interventions designed to emphasize 
the learning opportunities that come with behavioral 
missteps. Examples of non-punitive responses to 
behavioral infractions include peer mediation, conflict 
resolution, teacher conferences, and community 
service. Non-punitive practices therefore stand in 
contrast with punitive practices, which are intended 
to punish negative behaviors, and more specifically 
exclusionary practices—like OSS and expulsion—which 
remove offending students from the school. 


At the elementary and middle-school levels, SDP has 
worked to facilitate schools’ shift toward non-punitive 
disciplinary practices in a variety of ways. SDP has 
incorporated new content into principal and teacher 
professional development and offered training on 
conflict resolution to school leaders and staff ona 
voluntary basis. The district has also supported the 

use of evidence-based climate programs, specifically 
Positive Behavior Interventions and Supports (PBIS) 
and Restorative Practices (RP), and hired new staff at 
the district level to support these climate initiatives. In 
addition, SDP supports schools in identifying and hiring 
school-based climate staff—ranging from climate 
specialists who supervise lunch rooms and hallways to 
administrative-level climate managers. School leaders 
are quick to point out, however, that these positions 
must be funded by individual schools; there are no 
district funds to support them. 


Positive Behavior Interventions and 
Supports (PBIS) in SDP 


A focal point of recent climate programming in SDP 
is PBIS, a school-wide intervention designed to both 
improve overall climate by teaching and supporting 
appropriate behaviors for all students, and to provide 
more intensive supports to those students who need 
them (Horner & Sugai, 2015). The term PBIS is used 
broadly to refer to a set of principles and practices 
that can be applied at both the school-wide and 
the classroom levels. The model, which consists of 
three tiers of interventions, emphasizes consistent 
expectations, proactive encouragement of positive 
behaviors, and targeted supports. At Tier |, PBIS 
emphasizes explicit teaching and reinforcement of 
expectations for positive behavior. Tier | supports 
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are universal; that is, all students should experience 
consistent rules and rewards across all school settings. 
Students who require more support receive targeted 
assistance via small-group Tier Il interventions. Tier Ill 
consists of individualized interventions for students with 
serious behavioral challenges (www.pbis.org/school). 
In schools with effective PBIS programs, theory asserts, 
most students will respond to Tier | activities, with only a 
relative few needing Tier Il supports and a small subset 
of students progressing to Tier Ill (Bradshaw, Mitchell, & 
Leaf, 2010). 


Multiple rigorous studies have demonstrated the 
potential of PBIS to improve school climate, reduce 
behavioral incidents, and improve students’ 
socioemotional and academic outcomes (Bradshaw, 


Mitchell, & Leaf, 2010; Sugai & Simonsen, 2012; Vincent, 


Sprague, Pavel, Tobin, & Gau, 2015; Madigan, Cross, 
Smolkowski, & Strycker, 2016; McIntosh et al., 2014; 


Bradshaw, Waasdorp, & Leaf, 2012; Bradshaw, Mitchell, 


& Leaf, 2010). In response, more than 20,000 schools 
nationally have adopted PBIS (Horner, Sugai, Fixsen, 
2017). 


Since 2013, SDP has leveraged private and federal 
funds to provide PBIS training in more than 30 K-5, K-8, 
and middle schools. The number of schools receiving 
this training grew to 40 in 2017-18, representing a 
significant investment of district resources for the 
foreseeable future. While funding is a barrier to 
providing all, or even most, district K-8 schools with 
PBIS training and support in the short term, SDP has 
made the dissemination of positive behavioral support 
practices an explicit goal. 


Restorative Practices (RP) in SDP 


A second, less widespread district-supported 
initiative, RP, builds on the principles of restorative 
justice programs targeted to criminal offenders. The 
central goal of the approach is to focus students on 
reflecting on and mending the damage caused by 
poor behavioral choices (Gregory et al., 2016; Balfanz, 
et al., 2015; Wachtel, Costello, & Wachtel, 2009; 
Cameron and Thorsborne 2001), with an emphasis on 
relationships and community. RP is used primarily in 
high schools in SDP; however, some elementary and 
middle schools are also working to adopt aspects of 
the model. Only a handful of SDP elementary/middle 
schools have participated in formal RP training. 


In supporting schools’ adoption of programs like 
PBIS and RP, an SDP administrator explained, the 
district hopes to see broad changes in the cultures 
of its schools: “[We want schools to move away 


from] punishment without the learning and behavior 
change,” she said. “We want to make kids feel their 
learning environment is safer ... to make school a safer 
and kinder place for students.” 


An exploratory study of discipline and 
climate in SDP schools 


Prior research has established the potential of RP 
(Gregory et al., 2016; Gonzalez, 2015; Riestenberg, 
2013; Lewis, 2009; Jain, Bassey, Brown, & Kalra, 2014; 
McCluskey et al., 2008; Schiff, 2013) and PBIS (Madigan, 
Cross, Smolkowski, & Strycker, 2016; McIntosh et al., 
2014). However, as large, challenged urban districts 
like SDP look for solutions to the OSS problem, there 

is a pressing need for specific information about the 
difficulties these schools encounter and the supports 
they need to implement these programs well. There is 
still much to be learned about how these approaches 
can work—and what obstacles they face—in difficult 
contexts like SDP’s. Urban districts nationwide can 
benefit from new insights about the implementation 
of climate programming in contexts where punitive 
practices—including OSS—are entrenched (Eliason, 
Horner, & May, 2013) and often regarded as the only 
realistic option. 


In response to these gaps in the existing research, 

as well as SDP’s specific requests for insights and 
recommendations regarding district-level disciplinary 
policy and practices, this study was designed to 
answer the following research questions: 


1. What disciplinary practices are used by SDP 
schools serving students in kindergarten through 
eighth grade? 


2. What factors support or hinder schools’ alignment 
with the district’s climate and suspension- 
reduction goals? 


3. What patterns are evident in the approaches 
to discipline and climate of SDP K-5 and K-8 
schools? 


4. Are schools’ approaches to discipline and climate 
related to student disciplinary and academic 
outcomes? 


5. How are different approaches to discipline and 
climate manifested in individual schools? 


The following chapters describe the methods we used 
to explore these questions, and our key findings in 
response to each. 
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Chapter Two. 

The School District of 
Philadelphia’s Disciplinary 
Landscape 


The dearth of specific information about disciplinary 
practices in SDP schools is both documented in prior 
research (Steinberg & Lacoe, 2017) and acknowledged 
by district leaders. CPRE’s study therefore explored 
the variety of disciplinary practices used by SDP K-5 
and K-8 schools, and the beliefs that underlie them. 
Our goal with this work was two-fold: First, we aimed 
to develop a thorough understanding of the range of 
disciplinary practices schools are using in response to 
SDP’s emphasis on improving climate and reducing 
suspensions. Second, we hoped to identify key barriers 
and facilitators of schools’ alignment with SDP’s 
climate and suspension-reduction goals. The research 
described in this chapter thus addresses our first two 
research questions: 


Research Question 1: What disciplinary practices are 
used by SDP schools serving students in kindergarten 
through eighth grade? 


Research Question 2: What factors support or hinder 
schools’ alignment with the district’s climate and 
suspension-reduction goals? 


We addressed these questions via mixed-methods 
inquiry that included both surveys and in-depth 
qualitative research. This chapter describes our use of 
these methods and the major findings they produced. 


Research methods 


Qualitative research played an important role in our 
inquiry for research questions 1 and 2. The qualitative 
research was conducted over an 18-month period, 
between January 2016 and June 2017, and comprised 
three consecutive phases: 


Phase 1: A series of focus groups to identify critical 
issues for additional exploration, and to inform the 
development of survey questions. 


Phase 2: One-on-one interviews with the school-level 
staff who understand discipline in SDP schools most 
intimately. 


Phase 3: Field-based case studies to further explore 
the findings presented in this chapter as well as those 
discussed in Chapter 3. 


The methods and findings associated with Phase 3 

of the qualitative research—the field-based case 
studies—warrant a separate discussion and are 
detailed in Chapter 4 of this report. Here, we focus on 
the insights gained during Phases 1 and 2, and on the 
survey findings that explore them at scale. 


Qualitative research phase 1: Focus groups 


In January and February of 2016, CPRE researchers 
conducted four focus groups and three interviews 

with a range of school-level stakeholders, including 
teachers, assistant principals, counselors, school police, 
and special education teachers. We invited school 
staff members to participate in the focus groups via 
email, offering them a small incentive for participating. 
The focus groups were held after school, most often 

in the SDP central office building. On three occasions, 
only one participant arrived for a focus group. In those 
instances, the participants agreed to talk one-on-one 
with a member of our research team. The four focus 
groups that were held included one with school police, 
one with a group of special education teachers, 

and two focus groups which contained a mixture 

of individuals who held different positions at their 
respective schools. Focus-group discussions, as well as 
the three that were conducted as individual interviews, 
were guided by semi-structured protocols designed to 
probe variations in: 


e discipline processes used by schools, and the 
extent to which they are restorative or punitive in 
nature; 


e roles played in the discipline process by different 
administrators and staff; 


¢ culture around OSS, including perceptions of the 
utility of OSS; 


* resources schools use to implement their discipline 
approach, or wish they had access to; and 


e knowledge levels of school administrators and 
staff about school- and district-level discipline and 
suspension policies. 


All focus groups were audio recorded and transcribed. 
In formal memos, researchers detailed the major 
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themes of each focus group and specified propositions 
related to the topics listed above (Miles & Huberman, 
1994). These propositions were then organized across 
the set of researcher memos, yielding detailed 
information about the variations respondents described 
within these general topics. This overall understanding 
of how practices and approaches vary across school 
contexts informed the development of protocols for 
one-on-one interviews and items for teacher and 
principal surveys. In addition, the propositions and 
themes informed the findings detailed in this chapter. 


Qualitative research phase 2: Interviews 


Between February and June 2016, CPRE researchers 
conducted 81 one-on-one interviews with principals, 
climate staff, assistant principals, deans, and teachers 
in SDP K-8 schools. The interviews were conducted in 
person or by telephone using semi-structured, role- 
specific interview protocols. These protocols were 
designed to solicit in-depth reflections from participants 
in response to the following guiding questions: 


1. Are school administrators, teachers, and other 
school staff receiving the district’s message about 
the importance of reducing OSS and embracing 
non-punitive disciplinary practices? If so, by what 
means are they receiving this message? 


2. How do school administrators, teachers, and 
other school staff understand and articulate the 
district’s message? 


Table 1. 
Focus group and interview participants by role 


Principals 

Administrator-level Climate Staff 
Assistant Principals 

Deans 

Guidance Counselors 

Regular Classroom Teachers 
School Police 

Special Education Teachers 
Teachers of English as a Second Language 
Climate Coordinator 

Climate Support Staff 

PBIS Leads 


3. To what extent are school administrators, 
teachers, and other school staff philosophically 
aligned with the district’s message about the 
importance of reducing OSS? What are the 
beliefs and philosophies of those who are not 
aligned with the district’s message? 


What disciplinary practices are schools using? 


What challenges do schools face in implementing 
non-punitive practices and reducing OSS? 


Table 1 summarizes the focus-group and interview 
samples by role. 


The interviews lasted approximately 30 minutes and 
were audio-recorded and transcribed. Interview 
transcripts were coded for analysis by five members 
of the research team using Dedoose, a secure, cloud- 
based platform. Inter-rater reliability was established 
prior to coding; each researcher independently 
applied codes to the same transcript excerpts, and 
code applications were compared. Code definitions 
were refined to ensure clarity and the process 
repeated until all coders reached at least 80% reliability 
based on Dedoose’s reliability metrics. Once reliability 
was established, each interview was coded by one 
member of the research team. 


Broad codes derived from the guiding questions were 
applied to all interview transcripts initially. Examples 
of broad codes included: OSS philosophy and use; 
messaging about discipline; discipline/suspension 
process; disciplinary interventions; communication 


Focus Groups __ Interviews 
0 DD. 
0 1] 
1 2 
0) 10 
0 2 
2 30 
2 0) 
8 0 
1 0) 
] ] 

1 0 
0) 3 
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about discipline; and resources for discipline, among 
others. As the team applied these broad codes, more 
specific codes were developed inductively to address 
nuances in the data. For example, within data coded 
for schools’ discipline and suspension processes, finer- 
grained data emerged relating specifically to the role 
of paperwork and the Code of Conduct, the extent 

to which individual student factors are considered in 
decisions about suspensions, how data is used to inform 
decisions, and the role played by teachers and parents 
in the process. Within the broad code of Interventions, 
sub-codes were added for specific interventions such 
as in school-suspension and detention, as well as 

codes for delineating non-punitive from punitive, and 
proactive from reactive interventions. 


Once all transcripts were coded, each researcher 
was assigned to lead the analysis pertaining to one or 
more of the guiding questions by working to synthesize 
findings from the application of particular codes, 
across all transcripts. Researchers then wrote analytic 
memos distilling the content of codes in response 

to each of the guiding questions (Strauss & Corbin, 
1997), and identifying categories and themes (Miles 

& Huberman, 1994). Categories and themes were 
shared and discussed among team members in weekly 
meetings. These dialogic engagement exercises led 
to a sharpening of themes (Ravitch & Carl, 2016). 
Themes identified in this process were then compared 
and contrasted with insights from the survey data (see 
below), to distill the key findings presented in response 
to our first two research questions. 


Surveys 


Data were collected via online surveys administered to 
teachers and school administrators in all SDP schools 
during the spring of 2016. SDP’s Office of Research 

and Evaluation administers annual surveys to collect 
information about experiences and opinions of various 
stakeholders on a range of issues. To collect data 

for our analyses, CPRE researchers collaborated with 
SDP to embed items specifically developed for this 
study in the district’s principal and teacher surveys. 
These additional items were developed by the CPRE 
research team, and drew on prior research as well as 
insights and themes developed through the qualitative 
research described above. CPRE also worked with 

the district to identify a sample of teachers to receive 
another survey administered independently by the 
CPRE research team, in exchange for an incentive. This 
follow-up survey included an additional set of items 
that probed the topics of interest in greater depth 
than the SDP survey instruments permitted. More detail 
about the survey instruments and samples is provided 
in Chapter 3. 


SDP disciplinary landscape: Key findings 


This study’s first two research questions pertain to 1) 
the practices SDP schools use to address disciplinary 
issues and 2) the barriers they encounter in reducing 
suspension and improving climate. These questions 
arose directly from conversations with SDP leadership; 
having worked to address climate and OSS from policy 
and programmatic angles, district leaders sought 
answers to questions like: Are schools focused on 
reducing their reliance on OSS? If so, what are they 
doing instead of suspending misbehaving students? 
Are schools embracing non-punitive interventions? And 
if so, is this more true of some schools—those receiving 
district support for PBIS, for instance—than others? 
What do those approaches look like in practice? 
Similarly, they wondered, to the extent that schools 
may not be making these shifts, why are they not? 
What supports might help schools embrace climate- 
improvement initiatives? What barriers are in the way? 


In the discussion below, we detail the responses 

to these questions that arose from our research, 
organizing them into four key findings. Table 2 
represents these key findings as they pertain to the 
study’s first two research questions. 


Key finding #1: Seeking alternatives 


Principals we interviewed overwhelmingly reported 
that they actively seek alternatives to suspension when 
responding to behavioral infractions on the part of their 
students. One principal explained: 


Suspensions, in my opinion, are really the last 
resort, because obviously they're not effective in 
terms of long-term changes in behaviors and/or 
choices. To that end, we try to establish different 
types of consequences that we have a locus of 
control over in our school. 


Teacher survey responses corroborate principals’ 
accounts of their efforts to avoid the use of suspension 
in their schools, to some extent. However, they also 
highlight variation in the consistency with which 
principals communicate this intention. Overall, 64% of 
teachers responding to the surveys agreed or strongly 
agreed that their principals convey the message that 
“OSS is a last resort.” 


Teachers in the schools that are implementing PBIS 
with support from the district were significantly more 
likely than those in non-PBIS schools to report that their 
school leaders communicate that reducing OSS is a 
school-wide priority (78 vs. 64%), and that adopting 
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Table 2. 
Research questions 1 and 2, with key findings 


Research Question 


climate and suspension-reduction goals? 


school-wide alternatives to OSS is a priority (61 vs. 50%). 


This suggests that, while a majority of principals overall 
are communicating the need to reduce suspensions, 
this message is penetrating more fully in the schools 
implementing PBIS with the district’s support. 


Asked about the strategies they use in their efforts 

to avoid suspension, nearly all principals described 
responding to misbehavior with a tiered system of 
increasingly severe interventions. Many reported trying 
to incorporate non-punitive responses as part of this 
progression of consequences. 


The progressive system of disciplinary consequences 
principals and other school leaders described during 
interviews generally encompasses the full range of 
interventions available at a given school. The severity 
of the consequence for any particular infraction is 
determined based on two factors: the nature of the 
infraction and the extent to which it is a repeated 
offense by the same student. This approach is 
consistent with the tiered consequence system SDP 
advocates as part of its Code of Conduct, and is 

an outgrowth of the Response to Intervention and 
Instruction (RTII) and, more recently, the Multi-Tiered 
Systems of Support (MTSS) processes required by SDP. 
Schools in SDP are required to use and document a 
progressive series of interventions in order to justify 
requests for special services for students, including 
behavioral health and special education services 
(http://webgui.phila.k12.pa.us/offices/c/curriculum/ 
interventions/rtii2/offices/c/curriculum/interventions/ 


What disciplinary practices are used by SDP K-5 and K-8 schools? 


Key Findings 


Schools are seeking alternatives to suspension in response 
to student misbehavior. The extent to which this is an 
explicit priority varies across schools generally, and 
between PBIS and non-PBIS schools. 


A subset of schools is taking steps to proactively address 
student behavior through programming that encourages 
positive choices and engagement. This subgroup includes 
a combination of PBIS and non-PBIS schools. 


What factors support or hinder schools’ alignment with the district’s Administrators and teachers identify resource shortages 


in the areas of staffing, space, and supportive services 
as the biggest impediment to reducing the use of 
suspension. 


Administrators and teachers are often not aligned about 
how discipline should be managed and whether and 
when suspension should be used. This is the case in PBIS 
and non-PBIS schools. 


rtii/what-is-rtii3) . 


Teacher survey responses provide more detail on the 
specific disciplinary interventions their schools use. 
Table 3 displays the percentage of teachers reporting 
the use of particular interventions, and highlights the 
differences reported by teachers in PBIS and non-PBIS 
schools. 


These results indicate that schools rely on parent 
conferences more than any other single intervention 
in response to disciplinary infractions. One principal 
explained: 


If fhe parents are willing to correct the situation, 
and work with us, and are willing to be a part of 
the solution, we would be less likely to suspend 

the child. That’s what we want—the parents 
involved, the parents willing to help us change the 
behavior. 


We observed that the principals we interviewed in 

SDP K-5 and K-8 schools conceptualized students’ 
progressions through successively more intensive 
consequences in different ways. Some, for instance, 
describe the system as a series of strikes, with additional 
strikes accruing as students continue to act out. Others 
spoke more explicitly about the progressive system as 
designed to give students an opportunity to correct 
their behavior before receiving a suspension. However, 
leaders consistently characterized this progressive 
approach as a key tool in efforts to avoid suspending 
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Table 3. 


Disciplinary responses reported by teachers in SDP K-5 and K-8 schools 


Intervention 


Parent-teacher conferences 

Sending students to another teacher’s classroom 
Daily report for behavior 

Student-teacher conferences 

Individual behavior plans 

Lunchtime detention 

Out-of-school suspension 

School-wide system of incentives for good behavior 
In-school suspension 

EH-42 (Students sent home until they return with a parent) 
Activities to promote student reflection 

After-school detention 

Peer mediation 

Mentoring programs 

Community service 


Before-school detention 


Percentage Percentage of Percentage 
of teachers teachers in PBIS of teachers in 
reporting use schools (n=125) non-PBIS (n=525) 

reporting use reporting use 
a2 V3 92 
86 82 87 
84 82 84 
80 70* 82 
80 75 81 
77 80* 67 
75 74 ie, 
70 80* 67 
Si 38 42 
39 38 40 
36 42* 35 
31 32 26 
22 DS) 21 
19 26* 17 
13 15 12 
7 7 8 


* indicates statistically significant difference between responses from PBIS and non-PBIS schools 


students. School leaders often described suspension 


as a “last resort” to be used only once the school has 


“exhausted other options.” One dean said: 


Suspension—we don’t throw that word out there 
at our school. Suspensions are not a consequence 


that we really want to highlight or give out. It 
shouldn't be our first option. We want to take 
steps before we get to suspension. 


A notable result of this progressive system of discipline 


is that teachers are expected to use and document 
a series of interventions prior to referring a student 
to the administration. This represents a shift for many 


teachers, especially experienced teachers previously 


accustomed to “sending students to the office.” 


Although this shift has been underway for some time, 
our findings show that many teachers in SDP continue 


to resist the expectation that they manage most 
behavioral issues in the classroom. For instance, in a 
result that was consistent across PBIS and non-PBIS 
schools, 42% of teachers responding to our surveys 


agreed with the statement “My administration blames 


me when my students misbehave.” 


As we discuss in more depth later in this chapter 
and in Chapter 4, we find that this resistance 


leads to significant tension between teachers and 
administrators in some schools, and that this tension is a 
barrier to schools’ shifts towards non-punitive discipline. 


Key finding #2: An eye on climate 


While we find that most SDP K-5 and K-8 schools 

are seeking alternatives to OSS when identifying 
consequences for students’ misbehavior, a subset of 
school leaders we interviewed described efforts to 
reduce the need to suspend students by improving 
school climate overall. These administrators generally 
described their schools’ use of one or some 
combination of three strategies: 


e school-wide PBIS implementation as part of the 
SDP-supported PBIS initiative; 


¢ implementation of PBIS-type programming— 
particularly school-wide token economy systems— 
without special training or district support; and 


¢ implementation of other initiatives designed to 
build student engagement and investment in 
school. 
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School-wide PBIS implementation 


PBIS represents the most concrete and coherent 
climate initiative among K-5 and K-8 schools across 
SDP. In 2015-16—the year our survey and interview 
data were collected—30 SDP schools implemented 
PBIS with support provided by the district. These schools 
received training and implementation coaching 
through collaborative relationships between SDP and 
several different providers, whose coaches assisted 
schools in establishing PBIS structures and procedures 
like school-wide behavioral expectations and incentive 
systems. In addition, coaches have supported school- 
level PBIS teams in the use of a data system that allows 
detailed behavior tracking and intervention planning. 
According to the program model, each school’s PBIS 
team meets monthly to examine data and design 
action plans, with the goal of building strong processes 
for encouraging and rewarding good behavior. A 
principal explained: 


PBIS is a change of mindset. Its [goal is] to 
get students to actually understand what 
the expectations are. And where suspension 
is definitely consequence-based, PBIS is not 
necessarily consequence-basead; it’s more 
incentives and rewards. It is keeping things 
positive. 


School leaders in the SDP-supported PBIS schools spoke 
positively about the program and expressed the view 
that it has changed their schools for the better. As one 
principal explained: 


Our general philosophy starts and ends with PBIS. 
We are a PBIS school. We have worked really 
hard to make that a part of our culture. We 

have four rules—four expectations. Be prompt. 

Be polite. Be prepared. And be productive. My 
philosophy is that we teach and we reteach those 
expectations. We recognize kids and provide 
them with incentives based on whether or not 
they are meeting the expectations. If they are not 
meeting expectations, then we restore them as a 
part of a consequence around the expectations. 
So, that is where we are. 


Survey responses from teachers indicate some 
differences in the types of behavioral interventions 
used in district-supported PBIS schools, as compared 
with other SDP schools. As Table 3 illustrates, teachers in 
district-supported PBIS schools were significantly more 
likely than those in other schools to report the presence 
of a school-wide system of incentives for good 
behavior (80% of teachers in district-supported PBIS 


schools vs. 67% in other schools); the use of activities 
designed to promote student reflection (42% vs. 35%); 
and the presence of mentoring programs (26% vs. 17%). 
Teachers in non-PBIS schools were significantly more 
likely to report the use of student-teacher conferences 
(82% vs. 70%). 


Principals we interviewed in district-supported PBIS 
schools also emphasized the usefulness of the program 
for building collaboration and community among staff 
and students. One principal noted: 


[PBIS] works. It’s really good. It helps to shape the 
school culture and what our expectations are 

so that everyone understands that these are our 
core values. 


Similarly, responses from teachers in district-supported 
PBIS schools indicated an increased tendency for 
teachers to regard their schools as collaborative and 
cohesive. They were significantly more likely to report 
collaborating regularly with other staff about classroom 
management (64% vs. 51%), and to report that their 
principal communicates a clear mission for the school 
(50% vs. 38%). 


Despite these positive findings, progress in the PBIS 
schools has been slow. SDP’s own analysis reveals 
that, of the 20 schools found to implement Tier | with 
fidelity in 2015-16, only seven experienced decreases 
in their pre-PBIS suspension rates, and these decreases 
were small, ranging from .02 to .21 percentage 

points. Indeed, they were considerably smaller than 
the average annual decreases in suspension rates 

for all SDP schools from 2010 to 2015 (Wills, Karakus, 

& Wolford, 2017). In accordance with these findings, 
we observe that implementation of the model 

is Quite inconsistent. School leaders report that 
attaining school-wide consistency in enforcement of 
expectations and use of rewards is a major challenge, 
and that it is difficult to bring some teachers on board. 
Without consistent and uniform use of PBIS strategies 
by teachers, they say, students do not buy in to the 
program and its effectiveness is significantly diminished. 


For example, one principal reported that consistent 
implementation of PBIS by teachers was her school's 
biggest challenge, and that gaining teacher buy- 

in would be the key thing that would make their 
implementation stronger—even more so than financial 
resources. The issue, as she described it, is that many 
teachers don’t believe in rewarding students simply for 
doing the right thing. She explained: 


When it came to the token economy, the 
teachers didn’t buy into if. The conversation's 
happening [all the time], “Why | should | give the 
child a reward for doing the right thing?” 
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Additionally, a PBIS lead at one school also highlighted 
the impact that lack of teacher buy-in has ona 
school’s ability to implement the program as intended. 
She said: 


| feel that there is just foo much inconsistency. 

So, it’s not working the way it should. If we had 
consistency | think the system would have a 
chance to work better. But things aren’t consistent 
so | feel it’s kind-of like banging your head against 
the wall. So, all the things that! want to do are just 
not working. 


These challenges, too, are reflected in our survey data: 
For instance, teachers in district-supported PBIS schools 
were no more likely than those in other schools to 
agree that their schools do a good job of addressing 
disciplinary challenges proactively. 


Several factors may explain the difficulty of consistent 
PBIS implementation in SDP. First, SDP’s implementation 
of PBIS is phased. As a result, while some of the schools 
involved in our study had been working with the 
program for several years, others were in their first 

year of implementation and still struggling with PBIS 
fundamentals at the time of our data collection. Their 
schools’ implementations may look very different in 

a few years’ time. Second, the schools selected to 
implement PBIS were generally the most troubled in the 
district to begin with, serving neighborhoods with high 
levels of poverty and other challenges. As we discuss 
in more detail below, teachers and administrators in 
these schools report feeling overwhelmed by the needs 
of their students and the behavioral issues they exhibit. 
PBIS training and coaching alone, many report, are 
simply not enough. 


PBIS-like programming and other climate 
initiatives 


Schools that receive district-supported PBIS training 
are a distinct minority among SDP K-5 and K-8 schools 
overall. However, administrators of some other schools 
report adopting selected components of the PBIS 
model. In some cases, these administrators said they 
had learned about PBIS practices from schools that 
were part of the district initiative and wanted to use 
them as well. While they often lamented not having 
access to the resources provided to schools in the 
district initiative, these school leaders described 
doing their best to co-opt useful aspects of the PBIS 
approach—particularly, the token economy system 
and/or school-wide expectations—on their own. One 
principal explained: 


We have a PBIS model in the building that is self- 
funded. It is building-funded, so it is not through 

a grant. But, we do have a PBIS model. So, we 
talk about “Have you focused on rewarding the 
positive behaviors?” instead of giving attention to 
the negative behavior. 


Both interviews and survey data revealed other climate 
interventions used in smaller, but still notable, numbers 
of schools. These include mentoring programs, efforts 
to foster parent involvement, activities that encourage 
teacher-student relationship-building, and sports or 
other extracurricular programs. For instance, to address 
issues that erupt or begin on the playground, some 
schools have partnered with organizations that help 

to structure recess time to avoid activities that tend 

to lead to conflict. One assistant principal described 
this approach along with a program operated in 
partnership with the neighborhood police station: 


| think the fact that we're doing the socialized recess 
this year and giving kids the chance to get out there 
and be more actively involved, instead of sitting in a 
lunch room; | think that’s helped greatly. Also, we do 
have a great program which does gang resistance, 
stuff like that, with the Philadelphia Police Department. 
| think that’s been a big help. 


Additionally, some administrators said that they focus 
on climate and classroom management with teachers 
in staff meetings and grade group meetings. One dean 
recounted: 


We do a lot of classroom management work in 
our small learning communities, and conversation 
about that. We have a school-wide behavior 
incentive program where kids get tickets for 
positive behavior. We have fun Fridays, where 
they have tickets, and you can buy a fun Friday 
activity. But, if you don't have enough tickets then 
there's a re-teaching period. So re-teaching is one 
way we teach our expectations. 


Finally, schools have also made efforts to connect 

with parents, to make them more a part of the school 
community, and to form a partnership with them to 
help both parties work more effectively with students. A 
Climate Manager described his school’s approach and 
philosophy regarding working with parents: 


History has shown the more parental involvement 
you have, the more information you will know 
about the child. In turn, we may be able to help 
the parents with things going on at home and 
they get a sense of what is going on throughout 
the school day. So, we have our parent gathering 
once a month. 
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Key finding #3: Help wanted 


The third finding that emerged from our inquiry 

into research questions 1 and 2 is that schools feel 
limited in their abilities to implement alternatives to 
suspension by resource constraints. Shortages in three 
areas in particular, resoondents indicated, leave 
some schools with few alternatives to suspension for 
repeated infractions or offenses that might otherwise 
be handled within the school. These areas are: support 
staff for non-instructional areas of the school; space 
to accommodate students who need to leave the 
classroom; and mental and behavioral health services 
for students with issues the school is not equipped to 
manage. 


Most of the respondents we interviewed described 
shortages in one or more of these three areas as serious 
obstacles to the reduction of suspension and the use 
of non-punitive practices in their schools. However, 
these themes arose most distinctly in conversations with 
school leaders and teachers whose schools serve the 
neediest students—those located in neighborhoods 
most impacted by poverty, crime, and familial 
instability. These school administrators were quick to 
mention the inadequacy of their schools’ resources 

to address the range and intensity of experiences 
students bring to school, and the dire shortage of 
mental and behavioral health services for youth in the 
City. One principal stated: 


We have sick children who aren't geiting the 
services that they need from provider health 
agencies, let's just say. There’s a lot of neglect in 
the homes, and a lot of neglect even from [the 
Department of Human Services] not managing 
cases. 


A teacher added: 


There are severe mental health issues. | have six 
students that could be eligible for the [student 
therapeutic support program], where they would 
have a one-on-one adult. But either due to lack 
of evaluations, or how long it takes—l’m not sure— 
but these kids aren't getting the help they need. 
That is for sure. 


With limited access to services for large numbers 
of troubled students, teachers report struggling to 
manage behavior in the classroom. One teacher said: 


We have a lot of angry kids. Lot of angry kids. 
They’re angry because their parents are angry. 


Sometimes, the life at home’s bad, and | think we 
need more desensitizing. They come with a lot of 
anger, and | think it’s getting worse. 


Further compounding the challenges of behavior 
management in many Philadelphia schools, budget 
cuts have reduced most schools to a skeletal staff, and 
school closures have created space crunches in those 
that remain.? Schools’ attempts to create reflection 
rooms or any space inside the school building to de- 
escalate conflicts or offer students respite are thwarted 
by a lack of staff and space. A principal noted: 


We don’t have the ability fo do an in-school! 
suspension unfortunately. | don’t have enough 
staff. | don't have someboay to staff it. | don’t 
have a space for if. 


Another added: 


Number one limitation here: staffing. | have one 
dean for 1,500 kids. Staffing is really a huge, huge 
issue. Staffing and space. 


In the face of these challenges, school administrators 
shared that issuing an OSS can feel like the only 
option. With funds for more support staff, they report, 
accommodation or reflection rooms could be used to 
defuse conflicts without excluding students long-term. 
And, they noted “more eyes and ears” in hallways, 

in the lunchroom, and on the playground could be 
invaluable. One principal remarked: 


Just the presence [of support staff would be 
helpful], but also them watching. If they know 

that “Il watch this group of children every day for 
recess and these six students play together every 
day,” then if they see an outsider over there, that 
may be a problem. That may be something that’s 
about ready fo jump off. Or if you see a student 
who is usually playing football with Group A angrily 
walk away from football and go to the basketball 
court, that could mean that he and someboay in 
the football game have had a disagreement. Is 
this something that is going to resolve itself or is this 
something that needs somebody fo intervene? 
Someone needs to be able to say, “Hey, Student 
A and Student B had a big argument outside in 
the yard. A walked away, but they were still mad 
at each other.” [That support staff member] is your 
eyes and ears. 


Key finding #4: Not on the same page 


2 More information on SDP’s budget constraints is available at: http://thenotebook.org/articles/2017/03/23/philadelphia-s-school-budget-picture-remains- 


bleak-despite-surplus-this-year 
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The last key finding from our research on SDP’s 
disciplinary landscape pertains to our second research 
question, regarding the factors that support or hinder 
schools’ reduction of suspension and improvements 

to climate. Here, we find that school leaders’ and 
teachers’ alignment over discipline and climate goals 
is a critical factor. More specifically, we observe 

that while most principals believe that suspension is 
detrimental and agree with SDP’s goal of moving away 
from exclusionary practices, teachers are far less likely 
to express these views. 


In interviews, we asked principals about their beliefs 
regarding suspension and its role in their schools’ 
disciplinary approach. Most reported reticence to use 
OSS based, in part, on the recognition that suspensions 
often do not precipitate a change in behavior. As one 
principal explained: 


I'm just not a big fan of suspension. | understand 
according to our Code [of Student Conduct] 
some infractions warrant it. | just feel if you don’t 
have other interventions in place, what is really 
going to change? 


In addition to recognizing that a suspension on its 
own may not lead to behavior change in students, 
administrators said they work to avoid suspensions at 
their schools because students need to be in school 
to learn. Principals talked about “lost seat time” and 
remarked that students “can’t learn if they're not 
here.” An assistant principal described her school’s 
progressive approach as collaborative, with the aim 
of not having students missing school because of a 
suspension: 


Progressive [discipline] requires communication 
between administration, teachers, parents—so 
that everyone's on the same page and we're all 
going in the same direction. Ultimately, | would 
like to see any child miss not one day of school 
because of a suspension. 


Another assistant principal agreed, “My thought is 
the child needs to be in school. I'm really not for 
suspension.” 


Survey data from teachers paints a different picture. 
Overall—and in schools in the district’s PBIS program 

as well as those that are not—teachers expressed the 
overwhelming view that suspension plays an important 
role in maintaining order and ensuring student learning. 
Table 4 summarizes responses from teachers. 


In interviews, teachers and other non-administrative 
staff frequently described frustration with their 
administrators’ refusal to suspend students for what 


they regarded as serious or repeated offenses. One 
teacher articulated a common sentiment: 


We as teachers don’t really have a say-so in 
whether or not a child can be recommended for 
suspension. That's an administrative decision that’s 
made without our input. In some situations it’s fine, 
but in other situations, if leaves teachers really 
frustrated. 


Another teacher provided examples of behaviors that 
she believed should warrant suspension—or at the very 
least removal from the classroom: 


[There have been times when] | felt like that 

child needed to be out of the room. There were 
some things where children were being sexually 
inappropriate, and there were other situations 
where one child in particular was just non-stop 
cursing at the teacher, and harassing other 
students. | felt like it wasn’t being handled the way 
it should have been. 


Examples like this, in which teachers reported feeling 
isolated and unsupported, or even directly undermined 
by administrators, came up frequently in our interviews 
and focus groups. On the survey, nearly 40% of 
teachers—in PBIS schools as well as overall—disagreed 
with the statement that “my administration supports my 
decisions about discipline.” 


Non-administrative staff frequently attributed their 
principals’ reticence to suspend students to a 
preference for non-punitive disciplinary strategies—a 
preference they characterized as overly soft or 
inappropriate to the student’s infraction. A dean 
shared: 


| feel that there are times that [the principal] 
should put the suspension out there a little more. 
She’s a little bit “Let’s move if over, let’s have 
mediations.” Which, there's a role for that — but 
sometimes a message has to be sent. You cannot 
excuse certain behaviors. 


For their part, principals expressed an awareness 
of these differences, and their own frustrations with 
the challenges of bringing teachers and other staff 
on board with their climate goals and disciplinary 
approach. One principal shared: 


| think one of the challenges is you have the 
teachers that often want total consistency. And 
so the differentiation of consequences has been a 
very difficult conversation with those stakeholders 
because they don’t see the long-term effect. 

As the Chapter 4 case studies in Chapter 4 explore 

in greater depth, we observe that these differences 
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Table 4. 
Teacher views on out-of-school suspension 


Percentage of teachers in agreement 


(n=650) 

OSS is useful for sending messages to parents about the seriousness of infractions 89% 

OSS is useful for removing disruptive students so that others can learn 85% 

OSS helps ensure a safe school environment 84% 

OSS of misbehaving students encourages other students to follow the rules 81% 

OSS is useful as a deterrent to the suspended student's future misbehavior 64% 

The negative impacts of OSS outweigh any possible benefits 28% 

between school leaders and teachers have serious approaches to managing student behavior, and 
consequences; indeed, they can perpetuate student associations between school profiles and student 
misbehavior and undercut climate efforts. outcomes. Then, in Chapter 4, we contextualize all our 


findings through four in-depth, field-based case studies. 
Conclusions 


The goal of our research on the disciplinary 

landscape was to develop specific information 

about the key activities and issues in SDP K-5 and 

K-8 schools, through the lens of the district's shift 
toward climate improvement and suspension 
reduction. We learned that schools are changing their 
thinking about appropriate and helpful responses to 
student misbehavior. Some schools are also working 
systematically to implement preventative interventions 
in order to reduce the need for suspension. Predictably, 
schools face challenges in changing their approaches 
to discipline—both reactive and proactive—by a lack 
of resources. A key obstacle, and one over which SDP 
may have more control, is that principals and teaching 
staff often disagree about the appropriate use of 
suspension and the goals of climate initiatives. In our 
view, building consensus and alignment among alll 
adults in the school about discipline and climate goals 
and interventions should be a major focus at both the 
school and district levels. 


This landscape view is informative, but limited in what 

it can tell us about future directions for policy and 
practice in SDP. For instance, the general information 
here cannot tell us how these practices and themes 
are manifested in the contexts of particular schools. 
Nor can it tell us how they relate to student outcomes 
like suspension and academic achievement. The 
following chapters address these issues: First, in Chapter 
3, we present our work in identifying profiles of schools’ 
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Chapter Three. 
Profiles of Discipline and Climate 
in SDP K-5 and K-8 Schools 


Through the mixed-methods inquiry described in 
Chapter Two, we were able to develop specific 
findings in response to our first two research questions, 
which pertain to school disciplinary approaches and 
barriers to schools’ embrace of non-punitive practices. 
In this chapter, we discuss our use of survey data 

from administrators and teachers in SDP K-5 and K-8 
schools to conduct exploratory analyses in response to 
research questions 3 and 4: 


Research Question 3: What patterns are evident in 
SDP K-5 and K-8 schools’ approaches to discipline 
and climate? 


Research Question 4: Are schools’ approaches to 
discipline and climate related to student suspension 
and academic outcomes? 


Research methods 


To address research question 3, we conducted a 
statistical analysis to identify profiles of climate and 
disciplinary approaches in SDP K-5 and K-8 schools. 

In response to research question 4, we conducted 
separate analyses to examine the relationship between 
the school profiles and student disciplinary and 
academic outcomes. All analyses described in this 
chapter used survey data collected from SDP teachers 
and principals. 


Data sources: Teacher and administrator surveys 


Both research questions 3 and 4 were answered using 
data from two rounds of survey administration in Spring, 
2016. For the first round, CPRE embedded questions 
about climate and discipline in SDP’s principal and 
teacher surveys. These surveys are administered online 
annually to all principals and teachers in all SDP 
schools by SDP’s Office of Research and Evaluation. 
Questions developed for these surveys by CPRE 
addressed three general topics: Disciplinary Culture 
and Climate; Disciplinary Practices and Interventions; 


and Out-of-School Suspension. For each topic, items 
were developed to gather school-level information 
about disciplinary practices, communication and 
collaboration around discipline, and staff members’ 
personal beliefs. A total of 12 items from the SDP 
surveys were used for the analysis of school disciplinary 
approaches. The response rate for the district teacher 
survey was 57%. 


A second round of survey data collection was 
completed shortly after the initial round in order to 
probe the topics of interest in greater depth than the 
SDP survey instruments permitted. For this round, CPRE 
worked with the district to identify a stratified random 
sample of teachers who would receive another survey 
administered independently by the CPRE research 
team. We randomly selected one teacher per grade in 
each K-8 district school to receive the follow-up survey. 
Of 1,151 teachers in this subsample, 851 (74%) provided 
responses. A total of 21 of the items administered to 
teachers as part of this follow-up survey were used in 
the analyses described in this chapter. 


The final sample included responses from a total of 
3,776 teachers and 151 principals, representing 96.5% 
of SDP schools serving K-8 students. Forty-one items 
were used in the analyses described in this chapter. 
Table 5 illustrates the sources—whether SDP teacher or 
principal survey, or CPRE's follow-up teacher survey— 
and general topics of the survey items. 


Identifying profiles 


To answer research question 3, regarding patterns 

in SDP K-5 and K-8 schools’ approaches to discipline 
and climate, we applied a statistical method called 
latent-class analysis (LCA) to the survey data described 
above. LCA is an analytic method that can identify 
patterns that reflect underlying classes, or groupings, in 
multivariate categorical data (Dayton, 1998). Our goal 
in selecting LCA was to understand the extent to which 
the schools in our study fall into types, or profiles, based 
on their stakeholder-reported climate and disciplinary 
characteristics. For more information on the LCA , see 
Appendix A. 


School profiles and student outcomes 


In response to research question 4, we used event 
history analysis to examine the relationship between 
schools’ profile membership and their students’ risk 
of being suspended. Much of the research base 

on suspension has examined suspension ratios for 
students enrolled during a particular school year 
(U.S. Department of Education Office of Civil Rights, 
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Table 5. 
Number of survey items by instrument and topic 

Climate 
SDP teacher survey items 10 
SDP principal survey items 2 
CPRE teacher survey items 8 
Total 20 


2016; Gregory, Cornell, & Fan, 2011; Mendez, Knoff, 

& Ferron, 2002). While not typically used in studies 
examining suspension, event history analysis is an ideal 
method for this type of study. Using longitudinal data 
on when specific, relevant events transpired—when 
students enrolled in or transferred out of a school, for 
instance, or when they were suspended or absent—this 
method is able to provide more precise information by 
answering questions about the conditional probability 
of a student receiving a suspension during the period 
of time when she/he was actually attending a given 
school. The event history analysis student-level 
suspension data provided by SDP for the school year in 
which survey data were collected, 2015-2016. 

More information on the event history analysis is 
provided in Appendix A. 

Following the event history analysis of student OSS, we 
next explored the extent to which schools’ disciplinary 
profile membership predicts differences in student 
academic achievement. This analysis used 2015-2016 
data, provided by SDP, from the Pennsylvania state 
test, the Pennsylvania System of School Assessments 
(PSSA), for students in grades 3-8 in English, math and 
science (science is assessed in grades 4 and 8 only). 


The findings that emerged from these analyses are 
detailed in the remainder of this chapter. 


Key findings 


Table 6 summarizes our key findings in response to 
research questions 3 and 4. These are described in 
detail below. 


Key finding #1: Three profiles 


Three profiles—or types— were identified among SDP 
K-5 and K-8 schools as the best-fit solution to the LCA. 
In our confirmatory testing, we observed that schools in 
each of the three identified profiles had 99% average 
probability of membership in the profile to which the 
model assigned them. Moreover, we found that the 
average probability that any school would better fit 

a profile to which it was not assigned was less than 


OSS Practices Total 
2 0 12 
0 6 8 
7 6 2) 
9 12 4] 


0.01. There was no significant difference in average 
probability of class membership between the three 
groups (F (2,163) =0.35, p=0.7081). Viewed together, 
these results indicate a high level of confidence that 
the three profiles identified in the best-fit LCA solution 
were good representations of their constituent schools. 
Figure 1 shows the distribution of the 166 schools in our 
sample among these three profiles. 


Based on the magnitude and direction of the survey 
data in response to the 41 items that characterized 
each profile, we were able to develop descriptions for 
each. These are: 


Profile 1. Reactive and autonomous. Schools in this 
profile rely on punitive and exclusionary disciplinary 
responses to maintain order. Respondents tend to 
express the belief that OSS is an important means of 
keeping control in the school and protecting non- 
disruptive students from others’ misbehavior. Teachers 
in these schools perceive that they must fend for 
themselves with regard to discipline; administrators, 
they believe, will not consistently support them. 


Profile 2. Under-resourced and non-cohesive. Schools 
in this profile experience staffing and resource 
shortages most profoundly, and are inconsistent in 
their use of both punitive and non-punitive practices. 
Staff report low morale. Teachers report little 
collaboration around discipline, and feel blamed by 
administrators for their students’ misbehavior. 


Profile 3. Collaborative and relational. Schools in this 
profile are characterized by collaborative approaches 
to discipline, including frequent student-teacher 
conferences and the use of non-punitive practices. 
Morale is generally high; teachers feel supported by 
their administration. OSS is not viewed as an effective 
response to misbehavior. 


Figure 1 shows the distribution of schools across the 
three profiles. 


Survey respondents from schools in Profile 3 
(collaborative and relational) were more likely than 
other respondents to report feeling supported by their 
administrations and adequately trained to handle 
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Table 6. 
Research questions 3 and 4 with key findings 


Research Question 


What patterns are evident in SDP K-5 and K-8 schools 


approaches to discipline and climate? 


Are schools’ approaches to discipline and climate related 


to student suspension and academic outcomes? 


discipline. In addition, they were more likely to express 
the belief that their school handles discipline in a way 
that is collaborative, compassionate, and responsive 
to student needs. They more often reported the 

use of positive behavioral interventions (though, as 
noted below, they were no more likely than other 
respondents to teach in schools implementing PBIS 
with SDP support). These respondents were more likely 
than others to disagree with statements asserting the 
usefulness of OSS, and they reported that their schools 
used exclusionary practices less frequently than 
respondents in other profiles. 


Respondents in schools in Profile 1 (reactive and 
autonomous) were more likely than others to report 
the use of exclusionary practices. They were also more 
likely than others to agree with statements asserting 
the usefulness of OSS for maintaining order and 
discouraging future misbehavior, and to disagree with 
statements like the following: 


Key Findings 


Survey data suggest the presence of three types, or profiles, 

of K-5 and K-8 schools in SDP based on their approaches to 
discipline and climate: 1) Reactive and autonomous; 2) Under- 
resourced and non-cohesive; 3) Collaborative and relational. 


Schools exhibiting a collaborative and relational approach to 
climate and discipline were located in communities that had 
lower poverty, higher educational attainment, fewer households 
without English spoken in the home, higher percentage of White 
non-Hispanic residents, and fewer minors living with one parent. 


Schools implementing PBIS with SDP support are found in all 
three profiles, suggesting poor penetration of PBIS practices in 
many implementing schools. 


Controlling for demographics and other factors, students 
attending schools with collaborative and relational approaches 
(Profile 3) had the lowest risk of being suspended. 


Controlling for demographics and other factors, students 
attending schools with collaborative and relational approaches 
(Profile 3) had the highest academic achievement. 


¢ My administration supports my decisions regarding 
school discipline. 


e | know what is expected of me regarding student 
discipline. 


e |have been adequately trained to manage 
student behavior effectively. 


Finally, the data from respondents in schools in Profile 
2 (uUnder-resourced and non-cohesive) suggest that 
these schools’ disciplinary approaches are neither 
consistently punitive nor consistently non-punitive. 
Respondents in these schools tended to report low 
morale at their schools, and a lack of administrative 
support for their disciplinary decisions. They reported 
infrequent use of proactive or non-punitive disciplinary 
approaches, and tended to disagree with assertions 
that their school handles discipline in a collaborative, 
compassionate, or effective manner. Interestingly, 
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Figure 1: 
Distribution of SDP K-5 and K-8 schools among the three profiles 
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respondents in Profile 2 schools were less likely than 
others to report that their school has a guidance 
counselor. 


It is important to note that, although the LCA model 
described here produced a solution with a good fit 
relative to alternatives, the findings of our tests of the 
LCA model’s identification suggest that it is plausible 
that other profile solutions might fit the data nearly 
as well. One goal of this exploratory study is to test 
the suitability and utility of LCA for researching school 
climate and discipline. We do not regard the findings 
discussed here as confirmatory evidence of these 
profiles, but rather as one way of organizing complex 
data in a meaningful way within the broader mixed- 
methods study. We are interested in explicating 

the profiles based on the LCA of survey data in 
combination with case studies to better understand 
school discipline in context as it relates to student 
disciplinary and academic outcomes. 


Key finding #2: Neighborhood matters 


After identifying the three profiles, we explored 
differences between them on observable school and 
community characteristics. Table 7 presents aggregate 
statistics for student demographics in the three profiles. 
This is potentially important information, as differences 
in student outcomes across the three profiles could 

be solely, or in large part, attributable to differences 

in student population. Table 7 presents evidence that 
schools in Profile 2 most closely resembled the district 
overall. Schools in Profile 3 had the lowest percentages 
of students receiving lunch assistance and those 
identified as racial minorities, and fewer unexcused 
absences. These findings are statistically significant 
when comparing students in Profile 3 schools to the 
combined students in the other two profiles, and 
reveal the need to control for these differences in any 
analyses that compare student outcomes across the 
three profiles. 


In the case of neighborhood schools like those in SDP, 
community attributes may be an important factor to 
consider in understanding the climate and disciplinary 
approaches of the school. To better understand 
schools’ surrounding contexts and further investigate 
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Table 7. 
Aggregate student attributes by school profile 
Profile 1 Profile 2 Profile 3 All SDP 
reactive & under-resourced collaborative & 
autonomous & non-cohesive relational 

Student demographics 
Students with 2 1 OSS 9.8% 9.9% 8.7% 9.45% 
Limited English proficiency 11.0% 11.1% 10.5% 10.85% 
Female 47.3% 47.6% 48.3% 47.75% 
Free/reduced lunch eligibility 71.4% 68.4% 64.2% 67.74% 
Hispanic 23.3% 21.2% 16.4% 20.08% 
African American 53.0% 51.6% 48.5% 50.88% 
Grade 3.4 3.56 3.9 3.63 
School start to student entry (weeks) | 2 | 2 | 1.14 
Unexcused absences (weeks) 1.2 1.2 1.0 1.13 

Community Attributes 
Minors in family up to 100 FPL 42.4% 34.8% 28.4% 34.4% 
Minors living with one parent 66.5% 63.3% 54.2% 60.5% 
Non-English-soeaking home 27.2% 23.8% 19.2% 22.9% 
BS or higher education 18.1% 18.3% 27.0% 21.8% 
Hispanic 22.0% 16.0% 11.2% 15.8% 
White non-Hispanic 26.3% 27.9% 40.9% 32.7% 
Black non-Hispanic 43.5% 48.2% 38.6% 43.0% 
Asian non-Hispanic 6.2% 5.6% 6.6% 6.2% 
Other non-Hispanic 2.1% 2.3% 2.7% 2.4% 


Note: Grade, school start, and absences are reports as averages 


Table 8. 

Schools by Profile and SDP Network 

Network Profile 1 Schools Profile 2 Schools Profile 3 Schools Participating Schools 
perNetwork 

1 6 5 1S 

2 7 9 7 23 

3 7 2 |2 21 

4 8 6 20 

) 6 2 13 

6 2 6 11 19 

if D 9 18 

8 6 5 17 

9 4 6 15 

Total 48 50 67 165 
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Table 9. 


Estimates (with hazard ratio) for three event history models of OSS in SDP 


Limited English proficiency 
Female 

School start to stud entry (weeks) 
Unexcused absences (weeks) 
Grade 

Free/reduced lunch 

Hispanic 

African American 

Prior OSS 

Profile 1 

Profile 2 

Profile 3 


** D<.001 * p<.05 


differences that may exist between the three school 
profiles, we conducted additional analyses that looked 
at the composition of the surrounding communities 

for the schools in each profile. The demographic 

data used in this study were provided by the United 
States Census Bureau in the form of the 2011-15 
American Community Survey 5-year estimates for 
census tracts. School discipline profile data were 
joined to school buildings’ census tract shapefiles by 
building identification codes using the Geographical 
Information System (GIS) package from ESRI called 
ArcMap, version 10.2.2. This allowed demographic 
data reflecting the community context within which 
the school is located to be linked using a spatial join to 
the school buildings file. Each school was thus linked to 
an associated set of community variables pertaining to 
the population in its census tract. ¢ 


Just as we observed relationships between 

student demographics and school profile, we find 
statistically significant differences in the makeup 

of the communities surrounding schools in different 
profiles. Soecifically, schools in Profile 3 were located 
in communities that had lower poverty, higher 
educational attainment, fewer households without 
English spoken in the home, higher percentage of 
White non-Hispanic residents, and fewer minors living 
with one parent. Figures 2 and 3 show the prevalene of 
schools in each profile against a backdrop of poverty 
and race, respectively.’ 


Model | Model II Model Ill 

0.30 = (0.74) -0.30 ** (0.74) 
-0.42 ** (0.66) -0.42 ** (0.66) 
0.03 se (1.03) 0.03 eos) 
0.11 a (1.12) 0.11 ** (1.12) 
@,i|2 age (1.13) 0.12 (iS) 
0.29 at (1.33) 0.28 “* (1,33) 
0.56 & (1.75) OSS ** (1.74) 
0.93 “ (2.53) 0.93 “~ (2.52) 
1.68 is (5.39) 1.68 (8) ,657)] 

0.00 eS (1 0) 0.05 ‘ (1.06) 

-0.16 ** (0.85) — -0.06 * (0.95) 


Key finding #3: PBIS in every profile 


Our third key finding is that schools implementing 

PBIS with SDP support are found in all three profiles, 
suggesting poor penetration of PBIS practices in many 
implementing schools. Because of PBIS’s focus on 
communicating clear expectations and rewarding 
positive behavior, it would be expected that many 
schools implementing PBIS would be in the Profile 3 
(collaborative and relational). However, PBIS schools 
are represented in all three profiles, with more than half 
falling into Profile 1 (reactive and autonomous). Chi- 
square tests indicate that SDP-supported PBIS schools 
are as likely to fall into Profiles 1 or 2 as they are to fall 
into Profile 3. 


Figure 5 illustrates the geographic distribution of 
schools by profile. Our analyses indicate that there 

is no statistically significant geographic clustering; in 
other words, schools in all three profiles are dispersed 
throughout the city. 


As these analyses indicate—and perhaps not 
surprisingly—schools that demonstrate collaborative 
and relational approaches to discipline and climate 
are more often located in relatively more advantaged 
neighborhoods. However, they also reveal that this is 
not always the case. In Chapter 4, we provide detailed 
descriptions of schools working to improve climate 

and embrace non-punitive discipline against different 


3 Future analysis may involve looking at these demographic data by school catchment area or by a predetermined radius around the school building 
(i.e. average distance a student travels to school). However, the information presented in Table 7 looks only at the demographics of the nearest census 


tract. 


4 To protect the anonymity of individual schools, these figures cluster schools geographically, by SDP administrative network. 
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Figure 2: 
Distribution of school profiles over racial makeup of Philadelphia neighborhoods 
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Figure 2 (continued): 
Distribution of school profiles over racial makeup of Philadelphia neighborhoods 


Percentage of residents 
identifying as White 
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Figure 3: 
Distribution of school profiles and child poverty in Philadelphia 
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community backdrops, including those that are, based 
on these analyses, least likely to be home to Profile 3 
schools. 


Key finding #4: Profile and suspension 


To address research question 4, pertaining to the 
relationship between the profiles and student 
outcomes, we conducted two separate analyses. 
First we examined whether a student's risk of being 
suspended is related to his or her school's disciplinary 
and climate profile. 


We estimated a student's risk of receiving at least 
one OSS over the 2015-16 school year in three nested 
models, whose results are shown in Table 9. Model | 
represents the event likelinood of an OSS conditional 
on student demographic characteristics (see Table 7 
for district rates for all covariates), without factoring in 
school profile. The hazard ratios in Model | therefore 
represent the estimated relationships between 
students’ demographic attributes and the probability 
of an OSS in 2015-16, holding all other predictors 
constant. 


Figure 4. 


Model Il predicts OSS event likelinood, conditional on 
school profiles only. This model does not control for 
student characteristics. Results of this model therefore 
reflect the expected difference in a student’s risk of 
receiving an OSS, without accounting for the significant 
student-demographic differences between school 
profiles. 


Model Ill includes both school profile and student 
demographics. Model Ill is the preferred model 
because it controls for student risk factors, which are 
disproportionately represented in the three profiles of 
disciplinary approaches, when testing for differences 
in OSS event likelilnood. Table 8 presents parameter 
estimates and hazard ratios for each of these three 
models. 


A review of the results indicates that students attending 
schools in Profile 3 (collaborative and relational) have 
a significantly lower risk of being suspended than 
students attending schools in either of the other two 
profiles. This is the case when student characteristics 
are controlled for, and when they are not; when 
adjusting for student demographics, the difference 
between Profile 3 schools and the other two profiles is 
reduced but still statistically significant. 


Distributions of adjusted school-level OSS risk, by disciplinary profile 
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The differences between Profile 1 (reactive and 
autonomous) and Profile 2 (under-resourced and 
non-cohesive) schools in terms of students’ OSS risk 

are less clear. In Model Il, which excludes student- 
demographic control variables, the hazard ratios 
indicate no difference in students’ OSS risk between 
Profile 1 and Profile 2 schools. Controlling for student 
demographics, however, we find that students in Profile 
1 schools have a significantly lower expected likelihood 
of suspension than those in Profile 2. To illustrate the 
differences and variation in predicted suspension rates 
between profile groups once demographic controls 
are applied. Figure 4 overlays the distributions of the 
school-level model-adjusted probability that a student 
will receive an OSS during the 2015-16 school year for 
each disciplinary profile. 


The variation observed in school-level OSS rates 

within the district is present within each of the profiles. 
Focusing on the peaks of the distributions, however, we 
see that the average adjusted school likelinood that 

a student will receive at least one OSS in the 2015- 


Table 10. 


Total number of state assessments by grade and subject 


16 school year is lower (i.e. to the left) for Profile 3 as 
compared to the other two profiles. 


Key finding #5: Profile and academic achievement 


To further explore research question 4, we also 
examined the relationship between school profile and 
student achievement, as measured by Pennsylvania 
state assessments administered in third grade and 
above. Student-level assessment data for 2015-16 

in Math, English Language Arts, and Science were 
provided for this analysis by SDP. 


Tables 10 and 11 provide a description of the data 

we used for this analysis. Table 10 shows the number 

of 2015-2016 student assessments included in the 
analysis, by grade and subject. Table 11 provides a 
raw breakdown of the percent of students scoring 
proficient and/or advanced, by grade level, in schools 
in each profile. (Science is assessed in grades 3 and 8 
only.) 


Grade English Math Science 
3 10,503 10,696 
4 9,828 10,038 VOT9 
5 9,057 9,175 
6 7,842 TDW 
7 7,194 7,294 
8 6,883 6,989 6,931 
Total 51,307 52,111 16,910 


Table 11. 


Sample size and raw percent proficient/advanced by subject, grade and profile 


English Math Science English Math = Science 

Profile 1 2 3 1 2 3 1 2 3 
Grade 

$) 28% 25% 35% 20% 16% 26% 10,503 10,696 

4 25% 24% 32% 15% 13% 20% 40% 37% 46% 9,828 10,038 9,979 

5 25% 28% 36% 13% 12% 21% 9,057 iS 

6 24% 27% 44% 9% 9% 24% 7 842 7,919 

7 23% 28% 43% 8% 12% 24% 7,194 7 2D 

8 24% 30% 44% 8% 10% 20% 16% 22% 35% 6,883 6,989 6,931 

25% 27% 39% 13% 13% 23% 32% 31% 41% 51,307 = 52,111 16,910 


Note: 2015-2016 PSSA scores for K-8 SDP schools 


32 Consortium for Policy Research in Education | RR 2017 - 4 


Discipline in Context: Suspension, Climate, and PBIS in the School District of Philadelphia 


Table 12. 
Estimates (with standard errors) for three regression models of PSSA scores in SDP 

Model 1 Model II Model Ill 
English 
Intercept 1001.31 (1.69) 974.49 (0.79) 1016.48 (1.77) 
Grade 4.85 (0.25) 4.32 (0.25) 
Female 26.48 (0.82) 26.42 (0.82) 
Free/reduced lunch -25.69 (0.90) -25.37 (0.90) 
Limited English proficiency -87.84 (1.49) -87.49 (1.49) 
African American -55.4 (1.02) -54.77 (1.02) 
Hispanic -47.03 (1.23) -45.28 (1.23) 
Prior OSS -33.43 (1.19) -32.97 (1.19) 
Unexcused absences (weeks) -15.56 (0.34) -15.2 (0.34) 
2015-16 OSS -27.79 (1.33) D/O lne2) 
Profile 2 -37.86 (1.22) -18.83 (1.05) 
Profile 1 -36.28 (1.15) -24.19 (0.99) 
Math 
Intercept 1010.09 (1.69) 921.32 (0.80) 1025.43 (1.76) 
Grade =8,1)9 (O.25) -3.66 (0.25) 
Female 1.34 (0.82) 1.25 (0.82) 
Free/reduced lunch -23.19 (0.90) -23.41 (0.90) 
Limited English proficiency -67.21 (1.43) -66.71 (1.43) 
African American -71.21 (1.02) -71.05 (1.02) 
Hispanic -55.49 (1.22) -54.31 (1.22) 
Prior OSS -28.84 (1.20) -28.76 (1.20) 
Unexcused absences (weeks) -16.54 (0.34) -16.24 (0.34) 
2015-16 OSS -24.27 (1.33) -23.88 (1.33) 
Profile 2 -31.38 (1.23) -14.74 (1.05) 
Profile 1 -36.57 (1.16) -26.15 (0.98) 
Science 
Intercept 1504.01 (4.95) 1233.94 (2.48) 1526.44 (5.19) 
Grade -23.36 (0.66) -24.1 (0.65) 
Female -0.66 (2.50) -0.49 (2.51) 
Free/reduced lunch -41.11 (2.72) -4] 22 (2.72) 
Limited English proficiency -149.63 (4.32) -148.25 (4.31) 
African American -119.26 (3.13) -118.67 (3.12) 
Hispanic -87.04 (3.72) -84.67 (3.72) 
Prior OSS -60.9 (3.57) =O (357) 
Unexcused absences (weeks) -24.81 (1.04) -24.29 (1.04) 
2015-16 OSS -55.38 (4.06) -55.12 (4.06) 
Profile 2 -41.66 (3.83) -25.45 (3.21) 
Profile 1 -43.92 (3.57) -36.24 (2.97) 


Note: Profile 3 (collaborative & relational) is reference category 
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We compared group means separately for the three 
subjects using a model that regressed standard scores 
on school disciplinary profile, which was treated as a 
categorical fixed effect. To account for the clustering 
of student scores within school, a random effect 

was included in the model for school. In addition 

to the covariates used in the event history analysis, 
we included in this model an indicator for OSS. We 
included OSS as a control variable because we were 
interested in the effect of school disciplinary profile 

on achievement, above and beyond the effect on 
achievement of any suspensions students may have 
received. Differences between school profiles (i.e., post 


hoc multiple group comparisons between least squares 
means) were then estimated along with associated 
standard errors to test for statistical significance of 
group differences. Table 12 presents the results of this 
analysis. 


To assist with interpretation, we include Table 14, 
which presents standardized effects rather than raw 
differences. These effects were calculated as quotient 
of difference between least square means and the 
standard deviation of the dependent variable. 

As Tables 13 and 14 illustrate, after controlling for 
student demographics and prior suspensions, students 
enrolled at schools in Profile 3 (collaborative and 


Table 13. 
PSSA differences between school profiles 
Raw Mean Raw S.D LS Mean S.E. 

English 
Profile 1 937.6 96.9 944.9 0.79 
Profile 2 939.3 102.1 IS ORS 0.71 
Profile 3 977.7 20.2 963.7 0.68 

Math 
Profile 1 891.5 00.6 896.8 0.79 
Profile 2 886.1 98.1 885.4 0.70 
Profile 3 925.2 24.7 911.5 0.68 

Science 
Profile 1 1193.7 82.5 1200.2 2.42 
Profile 2 1193.3 187.1 1189.4 22 
Profile 3 1240.0 209.4 1225.6 2.07 

Note: In order to protect the anonymity of the participating schools, figures presented are approximate. 

Table 14. 

Standard effect sizes for estimated differences in PSSA scores by subject between school profiles 

Adjusted Group Mean Difference Standardized Effect 

English 
Profile 1 5.4 -18.8 0.05 -0.17 
Profile 2 -5.4 -24.2 -0.05 -0.22 
Profile 3 18.8 24.2 0.17 0.22 

Math 
Profile 1 11.4 -14.7 0.10 -0.13 
Profile 2 -11.4 -26.2 -0.10 -0.24 
Profile 3 14.7 26.2 0.13 0.24 

Science 
Profile 1 10.8 -25.4 0.06 -0.13 
Profile 2 -10.8 -36.2 -0.06 -0.18 
Profile 3 25.4 36.2 0.13 0.18 

Note: All differences statistically significant p<.0001; English SD = 109.68; Math SD = 111.02; Science SD = 196.03. 
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relational) scored, on average, 24 points higher on 

the PSSA English, when compared to students at all 
other schools. This is equivalent to a standard effect 
size of 0.22 SD. Students in Profile 3 schools also scored 
an average of 26 points higher in Math and 36 points 
higher in Science, for standard effects of .24 and .18 
SD, respectively. (The interpretation and implications of 
these effect sizes are discussed in Chapter 5.) 


Results of this descriptive analysis offer evidence that 
schools’ climate and disciplinary approaches are 
associated with differences in academic achievement. 
It is, however, plausible that student test scores could 
be a contributing factor in schools’ decisions about 
how to respond to behavior issues. Further research is 
needed to explore this possibility and also to identify 
the extent to which additional explanatory factors 
not included in this study may contribute to both 
schools’ disciplinary approaches and levels of student 
achievement. 


Conclusions 


Our work in developing the three climate and 
disciplinary profiles for SDP K-5 and K-8 schools is a 
centerpiece of our study, and one that reveals a 
number of useful insights. First, this research clearly 
suggests that there are meaningful differences 
between large groups of SDP schools that are relatively 
easily explained and understood. The teachers and 
school leaders we surveyed point out differences not 
only in the prevalence of punitive and non-punitive 
disciplinary approaches, but also in patterns of 
communication and collaboration, and in the support 
and morale of adults in the buildings. This is a promising 
finding, as it suggests that differentiating and tailoring 
supports for schools in each of the different profiles is 
both possible and potentially quite useful for improving 
approaches to managing behavior. 


Second, the findings of our suspension and academic- 
achievement analyses yield evidence of a relationship 
between climate and student outcomes. Students 

in Profile 3 (collaborative and relational) schools, 

we see, are suspended less often and achieve at 

a higher level academically than those attending 
other SDP schools, even when controlling for student 
demographics. This underscores other research 
(Voight, Austin, & Hanson, 2013) and suggests that 
district leaders looking to reduce suspension and boost 
achievement should look for ways to replicate the 
successes that are evident among these schools. Of 
course, we also see that the schools in Profile 3 are, 
overall, located in neighborhoods with lower poverty, 
crime, and proportionally fewer Black and Hispanic 
residents. This, too, underscores our finding—discussed 
in Chapters 2, 4, and 5—that, in general, the schools 


serving students with the most complex needs struggle 
most to effectively implement PBIS and non-punitive 
approaches to discipline. 


It is important to keep in mind, when considering 

the findings presented in this chapter, that this work 

is explicitly exploratory in nature. In addition to 
exploring new questions about patterns of disciplinary 
approaches and how they present themselves in survey 
data, we applied methods—specifically LCA and event 
history analysis—that are not typically used in studies 

of school discipline and climate. Future research 

should further establish the use of these methods in this 
context. 


An additional caveat to the findings discussed in this 
chapter is that, while evidence suggests that the three- 
profile solution has a good fit to the data, there are 
indications that it may not be the only reasonable way 
to define a disciplinary typology. This is important for 
two reasons: First, it invites more research to examine 
the applicability of these insights in other contexts. 
Second, it points to the need for a second perspective, 
one that looks more deeply at the profiles as they 

are manifested in actual schools. In the chapter that 
follows, which details our case study findings, we offer 
this additional view. 
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Chapter Four. 
The Profiles in Depth 


CPRE’s research into climate and disciplinary practices 
in SDP K-5 and K-8 schools culminated in four field- 
based case studies completed in the spring of 2017. 

A central purpose of the case-study research was to 
examine in depth how discipline and climate policies 
and programs are implemented in varying school 
contexts throughout the city. This work addresses our 
fifth research question: 


Research Question 5: How are different approaches 
to discipline and climate manifested in individual 
schools? 


The case-study research was intended to contextualize 
the results of the profile analysis described in Chapter 
3. By exploring a small number of cases in depth, 

we were able to better understand how various 
factors interact to contribute to a school’s profile 
membership. Finally, they offered a unique opportunity 
to explore how the district’s messages about OSS and 
climate have been received and enacted by various 
individuals within a single school. 


The four schools we selected for our case-study 
research occupy different Philadelphia neighborhoods 
with distinct socioeconomic profiles, and each 
contends with a unique set of challenges. For purposes 
of this research, we gave the four schools pseudonyms: 
Gannis Elementary/Middle; Downing Elementary; 
Stafford Elementary; and Clybourne Elementary/ 
Middle. Two of these four schools receive support for 


Table 15. 
The case study schools at a glance 

Gannis 
Climate & Discipline Profile 3 

(collaborative and 
relational) 

School size large 
Grades served K-8 
SDP-supported PBIS Yes 
% of students receiving a suspension in ~20% 


2015-16 


climate efforts through the district’s PBIS initiative. In 
two of the four, a student’s predicted likelinood of 
receiving at least one OSS during the 2015-16 school 
year was well below the district average. In one of the 
case study schools, a student’s average likelinood of 
receiving an OSS roughly equaled the district average, 
and students at another faced more than double the 
average risk of receiving an OSS for that year. Table 12 
displays additional information on each of the case- 
study schools. 


Despite their differences, these four schools are all 
working to implement climate strategies that align with 
the district’s goals, as compared with the population 

of K-5 and K-8 SDP schools overall. That is, each of the 
four schools is working to improve climate by adopting 
a relatively non-punitive orientation towards discipline. 
By focusing on four schools that purport to be generally 
aligned with SDP priorities, we were able to look closely 
at the strategies different schools use to achieve similar 
goals, as well as the specific challenges that arise in 
each context. 


Because of these differences and similarities, the 
insights we gathered from the case-study research in 
these four schools both affirmed and challenged the 
results of our other research activities. In this chapter, 
we present a brief description of each school. Our key 
findings from this portion of the study are presented at 
the end of the chapter, where we discuss our analysis 
of the four cases together. 


Research Methods 


To conduct each of the case studies described in this 
chapter, a team of two researchers spent at least two 
school days conducting observations, interviews, and 
focus groups on site. Interviews and focus groups were 
designed to represent a broad range of stakeholders 


Downing Stafford Clybourne 
3 1 3 
(collaborative and (reactive and (collaborative 
relational) autonomous) and relational) 
small medium medium 
K-5 K-5 K-8 
No Yes No 
<5% <5% ~15% 


Note: In order to protect the anonymity of the participating schools, figures presented are approximate. 
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and perspectives at each school. Observations 
focused on different settings within each school to help 
develop a clear sense of school-wide climate, and to 
better understand their individual contexts, strengths, 
and challenges. 


The specific structure of the data-collection visits and 
research activities varied based on each school’s 
circumstances and schedules. In each school, the 
principal and other administrators handling discipline 
issues were interviewed, along with classroom teachers. 
At two of the four, researchers also spoke with parents 
and students; for logistical reasons this was not 

possible in the other two schools. All interviews and 
focus groups were audio-recorded and transcribed 

for analysis. Observations of both structured spaces, 
such as classrooms and disciplinary spaces, and less 
structured spaces like cafeterias and playgrounds were 
also conducted. In each of these spaces, researchers 
compiled field notes using tailored protocols. 


The two researchers assigned to each school worked 
together to produce detailed memos highlighting 

key insights about the school’s approach to climate, 
discipline, and suspensions. The guiding questions 
presented in Chapter 2's description of interview data 
collection were modified to focus on a single-school 
context; these questions gave structure to both the 
collection of case-study data and the development of 
the research memos: 


1. Have people at this school received messages 
from the district about improving climate and 
reducing suspensions? 


2. Ifso, how do people at this school understand the 
messages they have received? To what extent 
are they bought in to these messages? 


3. Is this school responding to these messages? If so, 
how? 


4. How does this school manage discipline? To what 
extent is its qoproach proactive vs. reactive? 
Punitive vs. non-punitive? 


5. What barriers and facilitators exist to this school’s 
enactment of the district's vision? 


6. Are there signs of success in terms of positive 
climate and/or suspension reduction? If so, what 
are the signs and how do people in this school 
talk about them? 


7. Whatrole, if any, does PBIS play in this school’s 
approach to climate and discipline? 


The research team for each school reviewed drafts 

of the case-study memos while referring back to 
observation notes and transcripts. Summaries for each 
school were then shared with the entire qualitative 


team to discuss and clarify the frames and propositions 
suggested by the team of researchers from each 
individual school (Miles & Huberman, 1994). Through 
inter-team discourse, the propositions were sharpened 
into a set of findings particular to each case. As a final 
step, the team compared and contrasted the findings 
and insights of the four cases in light of the guiding 
questions listed above and research question #5. 
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Case study 1: A diamond in the 
rough 


Floor-to-ceiling windows frame glass doorways 
on either side of Gannis Elementary & Middle 
School’s® main corridor. The school’s facility is 
relatively modern, with colorful student artwork 
filling glass cases on the walls. The feeling in the 
K-8 school’s main artery is welcoming, belying 
Gannis’ location in a particularly troubled area 
of Philadelphia. As a class passes through, 

a few students exchange greetings with a 
support staff member who is seated at the 
reception desk—a long-time Gannis employee 
with deep roots in the neighborhood. 


Demographically, Gannis’ surrounding 
neighborhood is predominately Hispanic; the 
school’s principal, Ms. Rodriguez, reports that 
more than 80% of the students are Hispanic, 
and many are classified as English-language 
learners. African Americans constitute the 
second largest population of neighborhood 
residents, and Rodriguez noted a growing influx 
of Asian and white students to the school, 
although their numbers are still small. 


Generally soeaking, Gannis’ leaders and 

staff expressed the belief that the school’s 
commitment to cultivating positive relationships 
among students, families, and staff yields 
benefits in student cooperation, buy-in, and 
behavior. Parents acknowledged the school’s 
open-door policy and receptiveness to their 
participation. Though Gannis receives support 
for PBIS implementation from the district and 
its student-centered climate philosophy is 

well aligned with the program, staff members 
characterize the school’s climate as a 

natural outgrowth of staff's connection and 
commitment to the local community. 


“We as a team try to just work on creating 
a family environment,” Rodriguez 
explained. “We are very receptive to 
families coming in.” 


Over the course of three separate visits to 
Gannis during May, 2017, CPRE researchers 
conducted interviews and facilitated focus 
group discussions about the school’s climate 


Discipline in Context: Suspension, Climate, and PBIS in the School District of Philadelphia 


and disciplinary practices. Allin all, we 
collected insights from the principal; assistant 
principal; dean; counselor; eight teachers; two 
support staff members; a bus driver; the school 
resource officer; five parents; and 12 students. 
We observed interactions among students, 
staff, and family members in instructional and 
non-instructional spaces throughout the school. 


The portrait that emerged in the course of 

this research is of a school that faces serious 
challenges in terms of student needs and 
behavior—its student body is among the most 
challenging in our case study sample—but that 
is nonetheless held in high regard by its staff 
and the local community. Gannis highlights and 
celebrates its students’ predominantly Hispanic 
heritage; the school’s senior leadership team is 
almost entirely of Hispanic descent, and many 
staff members claim personal connections 

to the surrounding community. At least one 
teacher at each grade level is a native Spanish 
speaker. The school also has strong relationships 
with a number of community organizations that 
support Gannis’ staff and bring extra resources. 


Gannis also emerges from our case-study 
research as a school whose staff members 
appreciate its strong leadership and embrace 
a coherent climate vision—specifically, 

a student-centered and generally non- 
punitive approach that is tempered by zero 
tolerance for serious behavioral infractions. 
This finding is consistent with other data from 
the school: Survey respondents reported using 
predominantly proactive and non-punitive 
disciplinary practices. Yet, Gannis’ OSS rate 

is more than double that of any other school 
in our case-study sample, and that of SDP 

K-8 schools overall, reflecting the serious 
challenges which the school continues to face. 


High expectations as a foundation 


Despite important assets—including deep 
local connections; strong community partners; 
and a shared vision for what the school can 
become—Gannis has faced and continues to 
face real and evolving challenges. Students 
grappling daily with serious issues like poverty, 


5 All names in this chapter are pseudonyms, and other identifying details have been omitted or changed. 
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homelessness, and crime bring significant 
behavioral challenges to school, and staff 
members feel underequipped to address their 
students’ needs. Virtually all stakeholders we 
spoke with, including parents and students, 
stated that staff shortages made behavior 
management a challenge at Gannis. “We 
need definitely more eyes and ears in the 
hallway,” a teacher reported. 


The school’s challenges were exacerbated 
when enrollment more than doubled following 
the closure of a nearby school several 

years ago. Staff members reported that the 
influx of new students led to an increase in 
problem behaviors and that, although the 
school received additional teaching staff, no 
additional administrative supports or services 
were provided to ease this transition. By many 
accounts, the closing of the nearby school and 
Gannis’ absorption of its students represented 
a turning point after which the school’s culture 
had to be recalibrated. 


Rodriguez assumed a lead role in that 
recalibration. She reported demanding that 
teachers have high expectations of Gannis 
students, and that they be strong managers 

of student behavior in their classrooms rather 
than relying on administrators to address lower- 
level offenses. Thus, a significant component of 
Gannis’ current strategy for improving climate 
is hiring individuals with a mindset that reflects 
the principal's “high expectations” mandate, 
a willingness to go above and beyond in 

their job, and a belief in relationship-building 
as a strategy to effectively manage student 
behavior. Teachers corroborated that success 
at Gannis requires a stance of personal 
accountability for classroom management, 
noting that some who fail to demonstrate 
commitment have been moved out of the 
school. 


In part because of Rodriguez's emphasis on 
teachers managing behavior at the classroom 
level, behavior management is an area where 
many Gannis teachers feel they need more 
support. As one explained: “We need more 
support for [behavior management]. Especially 
... 'd say new teachers but even myself, and 
I’ve taught for [a number of] years.” 


Gannis teachers reported that while SDP offers 


training and resources in this area, attending 
the sessions that are offered outside school 
hours can be inconvenient and the usefulness 
of the offerings is inconsistent. As a result, many 
Gannis teachers reported learning classroom 
management strategies from their colleagues 
or from training received during their tenures at 
charter schools. 


Rodriguez’ efforts to build a strong and 
committed teaching staff appear to be 
working: Teachers at Gannis were quick to 
praise the school’s professional environment, 
and characterize the school as “a diamond in 
the rough”—a special place with the potential 
to be even better. Teachers’ commitment to 
improving the school is evidenced not only 
through their descriptions of the school climate 
but through their active efforts to contribute to 
its future, such as writing grants to benefit the 
school’s students and pursuing programs and 
additional resources. 


Along with high expectations for her staff, 
Rodriguez used clear messaging early in her 
term to underscore behavioral expectations 
for students and families. Soon after her arrival, 
she tightened the school’s uniform policy, a 
change she identifies as the start of the process 
of reshaping the culture and equating the 
school, in some parents’ minds, with charter 
schools in the area. “We knew that for a lot 

of the parents it’s all about perception and 

it’s all about what [the school] looks like,” she 
reported. 


Staff members’ comments suggest that 

the new administration took swift action to 
correct student behavior by instituting high 
expectations. Further, if the principal said she 
was going to deliver a particular consequence, 
she followed through, which teachers 
appreciated. 


Gannis personnel report that the school 
exhausts a variety of other consequences prior 
to suspending a student when the infraction 

is less severe. Lunch detentions are common, 
and because the school cannot staff an in- 
school-suspension room, misbehaving students 
may be assigned to “shadow” the counselor 

or dean for a day. Suspension is not regarded 
by Gannis’ staff as an effective strategy for 
impacting students’ behavior. However, Gannis 
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contends with significant numbers of serious 
behavioral infractions, such as fights, and the 
school’s administration does not hesitate to 
issue an OSS when they believe it is warranted. 
In those cases, the dean explained, staff 

are careful to maintain proper paper trails 

so that, if necessary, district-level support for 
suspensions or even disciplinary transfers can 
be assured. Rodriguez explains: “We have 
reduced suspensions, but in order to change 
our climate we had suspend more just to make 
a change, just to get the parents to understand 
we're zero tolerance and that’s what we want 
to be. Then once parents started getting it, the 
kids started getting it.” 


Climate is relationships 


Although Gannis is well along in its participation 
in the SDP PBIS program and staff emphasize 
their efforts to support students’ positive 
behavior, most of those we spoke with 
characterized the school’s approach to 
discipline as based in caring and relationships 
rather than one that is specifically aligned 

with PBIS. Indeed, Gannis is not immediately 
recognizable as a PBIS school. For instance, 
the token economy system popular with many 
PBIS schools is Used only sporadically and by 
individual teachers at Gannis. Staff attributed 
the sparse use of PBIS-specific practices 

both to the lack of a PBIS point person at the 
school, and to leadership's more general 
emphasis on climate and relationships. Middle- 
school teachers generally reported that the 
incentives-based approach espoused by PBIS 
has not proven effective with their students. 
Instead, they focus on high expectations for 
student behavior, treating students as young 
adults, and developing mentoring relationships 
and personal connections with students. Staff 
described helping students develop intrinsic 
motivation for acting appropriately in school, 
as opposed to using the extrinsic motivators 
they associate with PBIS. 


Gannis’ principal asserted that staff members 
frequently give students options as part 

of making good decisions and provide 
opportunities for students to develop positive 
relationships with role models in the school. 

A younger student who struggles with lunch 
room behavior, for example, might be 


paired with an older peer mentor during 
lunch. The assistant principal described using 
extracurricular opportunities as a way to 
build relationships between students and the 
adults in the building. He cited taking time 
out to play basketball with the boys as a 
way to build positive mentoring relationships 
with the students that would lead to better 
communication and cooperation at school. 
Gannis staff claimed that this relationship- 
building is at the heart of all the school’s 
climate goals. 


In sum, this “diamond in the rough” is led by a 
principal's evident commitment to behavior 
and climate, with or without PBIS. She said: “We 
never have our hands off climate. Climate is... 
that’s like my baby. That’s where my work is aft. 
My heart is in the climate.” 
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Case study 2: “We’re a family” 


According to staff, Downing Elementary has 
enjoyed a strong reputation in its community 
for decades. A small school serving pre-K 
through fifth-grade students, Downing occupies 
an old but well-maintained building that has 
been creatively embellished in recent years, in 
keeping with the increasingly artsy feel of the 
surrounding community. It is brightly decorated, 
with spotless floors and student work adorning 
the walls, and is relatively well-resourced, with a 
library, a computer room, and a well-equipped 
gymnasium. Other than teachers’ and students’ 
voices emanating from classrooms, the 
hallways are generally quiet. 


Teachers and staff describe the climate 

at Downing as exceptionally positive, and 
grounded in the belief that all members of 

the school community are “family.” This notion 
of family is reinforced through the active 
promotion of norms that emphasize respect. 
Contributing to this cohesiveness is the fact that 
many employees, including the principal, have 
roots in the local community. Many of the staff 
members grew up in the neighborhood, have 
taken up residence in the community, or have 
enjoyed a long tenure at the school. Some 
students are the children of former Downing 
attendees, which, staff said, deepens the 

trust and familiarity between families and the 
school. According to a veteran teacher, “kids 
really enjoy coming to school here.” 


Situated in a Philadelphia neighborhood that is 
quickly gentrifying, Downing is aptly described 
as an “evolving” Community. School personnel 
described the neighborhood's newer residents 
as more “affluent” and “professional” than 
those native to the area. Many new residents 
engage with the school, some in preparation 
for their as-yet-unborn children’s eventual 
enrollment. While Downing staff reported that 
these residents bring an array of resources 
(i.e., Connections and funds), the principal, Ms. 
Peters, reports mediating emerging conflicts 
between neighborhood newcomers and 
members of the school’s established Home and 
School Association. At times, she explained, 
these groups have “different perspectives on 
what they think their community should look 
like.” 


In interviews with 11 staff members—the 
principal; guidance counselor; two parents; 
one bus driver/special assistant; and six 
teachers—we consistently heard about the 
gradual push-out of long-time neighborhood 
residents amid the rising price of local housing. 
Despite the transformation occurring in the 
neighborhood, stakeholders agree that 
Downing has continued to maintain a close-knit 
family feel that is characterized by a high- 
level of parental engagement. Class sizes are 
relatively small at Downing, with fewer than 25 
students per room. 


Downing’s student population is about 60% 
Caucasian, 15% Latino, and 10% African- 
American. Survey data collected from staff 

as part of this study suggest that the culture 
around student discipline is collaborative and 
student-centered. Additionally, Downing staff 
members were less likely to report the use of 
punitive disciplinary practices than teachers 
at any of the other three case-study schools 
described in this chapter. In conversations, 
teachers reported feeling fortunate to work 
there. 

The administrative team at Downing is small— 
consisting of just Peters and a guidance 
counselor. Each Is highly visible throughout the 
school. During our researchers’ visits, students 
approached the administrators frequently to 
say hello or to report on how they were doing. 
Discipline referrals at Downing are handled 

by the principal, often in consultation with the 
guidance counselor. The counselor, however, 
admitted that she feels conflicted about her 
secondary role as disciplinarian because of its 
potential to undermine the trusting relationship 
she needs to support students with personal 
issues. 


Downing is not a school with chronic discipline 
or climate issues; however, teachers report that 
there are instances when suspension should be 
used. In contrast, the principal and guidance 
counselor avoid Issuing suspensions at all costs. 
Staff reported that discipline and suspension 
practices were quite different under prior 
administrations. One veteran teacher remarked 
that Downing was once “kind-of run like a 
Catholic school...everything was cut and dry... 
strict” —thus suspensions were frequent. In our 
interviews, teachers characterized disciplinary 
incidents as more isolated and less severe than 
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in the past. Nonetheless, they also perceive the 
current discipline strategy as overly lax. 


Competing ideologies 


According to teachers and administrators at 
Downing, reducing suspension has been a 

goal of the last two successive principals. The 
current administration reported being aware of 
the District's emphasis on reducing suspensions, 
and that this past year the school’s OSS rate fell 
to nearly zero. In 2015-16, the year our survey 
data were collected, the OSS rate was less 
than half that of District K-8 schools overall. 


Yet, despite Downing’s solid culture, it is not 
without its challenges. The virtual elimination of 
suspension as a disciplinary option has created 
great frustration among teachers, and thus 
brought to the fore ideological differences 
between teachers and administrators. 
Although the administrative team embraces 
the district’s philosophy, teachers’ beliefs 
about the use and purposes of suspension 
have been slower to evolve despite their 
acknowledgement that suspension may not 
actually lead to positive changes in student 
behavior. Teachers reported that suspending 
students sends a strong and sometimes 
necessary message to students and families 
about moral and behavioral expectations. In 
addition, they acknowledged that an OSS gives 
teachers a needed “vacation” from a child’s 
antics. 


Teachers mentioned struggling with Peters’ 
message that teachers should handle student 
behavior on their own rather than sending 
students to her and the guidance counselor. 
According to the guidance counselor, teachers 
are told to “try to address [student infractions] 
using whatever skills they have. If it’s a student 
that has issues that continue to arise—a 
frequent flyer—teachers are to come up with a 
plan.” 


This represents a significant shift in the way 
discipline is handled at Downing. According 
to the guidance counselor, pink slips were 

“a big part of the culture” as recently as the 
2015-16 school year. In fact, Peters described 
the situation as having pink slips coming “out 
of her nose.” According to teachers, the 


option of referring discipline matters to the 
office allowed them to focus on instruction. As 
a result of administrators’ expectations, pink 
slips decreased dramatically in 2016-17. The 
principal reported that while Downing’s staff 

is aware of the Code of Conduct, it is not a 
focus at the school. She described this tool as 
inconsistent with the Downing philosophy of 
dealing with student discipline in a non-punitive 
way. 


Downing teachers recounted incidents in 
which their calls for a student's suspension 
went unsupported by the principal. The 

result, they said, was that teachers feel at 

odds with administration. Though somewhat 
grudgingly, they report that they do now 

work harder to handle behavioral incidents 

on their own. When disciplinary referrals do 
take place, teachers at Downing report that 
their administration’s preferred method of 
handling infractions consists of conversations 
with students to discuss their infractions. One 
teacher complained, “Sometimes there’s not a 
lot of consequences for disruptive behavior. It’s 
more of a conversation, which is fine, but then 
sometimes | feel like it needs to be taken a step 
further.” 


Current approaches to discipline 


Through our interviews with the principal 

and various members of the school staff, 

it was clear that Peters deeply believes 

that suspension is an ineffective response 

to handling student infractions, and that 

she objects to punitive consequences for 
children. Rather, she professes a philosophy 

of proactively working to reduce disciplinary 
issues, and subsequently curtailing the need to 
suspend students from school. Thus her strategy 
is to emphasize conflict resolution, respect for 
peers, an atmosphere of inclusion, and positive 
community norms. She reported: 


I’m not of a punitive nature about 
suspensions. Suspensions don’t work... 
there’s a mentality behind it that people 
believe if somebody does something so 
egregious they need to be suspended 
or if sends a message fo the parent, it’s 
going to stop the behavior. But then it 
never does ...So, I’m of the mindset that 
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we need fo come from a positive place 
first. Point out what people are doing 
right, what students are doing right. 


The guidance counselor, whom the principal 
hired, is closely aligned with this philosophy. 

In narrating an incident involving a student 
from a troubled home, she described herself 
as “coming from the emotional side of it and 
really wanting to treat her wholly, versus being 
punitive and saying ‘You knew what you were 
supposed to do, so it’s detention.’” 


In our interviews with school personnel, we 
noted plenty of evidence of Peters’ message 
of using positivity to shape the school’s culture. 
For instance, a parent volunteering in the 
lunchroom described the principal’s wish “to 
keep it a positive environment...she’s not a 
fan of yelling or that type of discipline... just a 
respectful environment between the kids and 
the adults.” 


To achieve these goals, the principal and 
counselor at Downing have instituted an online 
behavior-tracking system that transmits daily 
updates to parents; and an incentive system 
that rewards students for good behavior with 
fun activities on Fridays. Unlike many schools 

in the district, Downing has adequate support 
staff to assist with discipline; the principal 
reported that she is able to send support to a 
classroom when a teacher needs it. 


Peters acknowledged teachers’ criticisms that 
not all students respond to this positive, non- 
punitive approach. In more challenging cases, 
the counselor works to identify interventions 
for individual students, often with the goal of 
helping students understand what is motivating 
their own behavior. In addition to talking with 
students, the counselor frequently contacts 
parents and invites them in to discuss issues 
both at school and at home. The guidance 
counselor may also initiate the RTIl process, 
triggering interventions like a daily behavior 
report with frequent teacher sign-offs. At the 
time of our data collection, roughly 4% of 
Downing’s student body had been assigned 
behavior reports or other RTIl interventions. 


Students with ongoing behavior challenges 
may lose their Friday privileges, and may be 
required to participate in service learning (i.e., 


helping in younger students’ classrooms) as a 
form of restitution. Peters believes this not only 
serves as a consequence for students, but also 
helps to build empathy; teachers noted that 
service learning may serve as a reward rather 
than a punishment since students typically 
enjoy it. In addition, students with ongoing 
disciplinary issues must spend time engaged 

in reflective activities such as writing apology 
letters to those they may have wronged. These 
reflection activities take place in a designated 
room that is overseen, a few hours a week, 

by a teacher who has a reputation for being 
“strict” and who holds a de facto disciplinarian 
role for the school more generally. Both the 
teacher and the principal acknowledge that a 
“good cop, bad cop” dynamic exists: Students 
are referred to the de facto disciplinarian by 
teachers who believe the principal would 
administer a less stringent consequence than is 
warranted. 


In sum, Downing teachers see that Peters is 
trying to be preventative and to deal with 
problems that arise on a case-by-case basis. 
They appreciate the positive climate her 
approach helps support. At the same time, 
they openly wish for a stronger approach to 
discipline. For her part, the principal is aware 
that teachers would prefer she use suspension 
more and that she’s perceived as being “soft” 
on students. Nonetheless, she is committed 

to her vision and to encouraging teachers 
and staff to come up with alternative ways of 
managing behavior. 
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Case study 3: A grounded approach 


At 8 a.m. on a Wednesday in April, Stafford 
Elementary School students slowly trickle into 
the cafeteria as adults greet them with cordial 
“good mornings.” One student settles at a 
table where three others play Connect Four. 
Others line up to retrieve their breakfasts from 
cafeteria workers chatting over the sounds 

of a radio. Out on the playground, at least 

30 adults—mostly parents—linger as perhaps 
A400 kids talk in groups or engage in horseplay. 
Eventually, the students begin to form lines 
and ultimately follow their teachers through 
the door and down orderly corridors lined with 
inspirational quotes and artwork. 


This morning routine underscores Stafford’s 
overall feel: It is a place with established 
routines, attentive staff, engaged parents, 

and rules and procedures that are widely 
understood and generally observed. It is home 
to approximately 500 K-5 students. In 2016-17, 
the student population was just under 50% 
Caucasian, around 20% African-American, and 
15% Latino, with the remaining student body 
made up of Asian students and those of other 
ethnicities. The school had an average daily 
attendance rate of 95% in 2015-16. About three 
quarters of Stafford students are considered 
economically disadvantaged—a far lower 
percent than in SDP overall. Academically, 
Stafford’s fifth-grade students scored above 
the district average on both reading and 

math on the 2016 PSSA. The surrounding 
neighborhood is quiet and largely residential, 
and boasts several playgrounds, hiking trails, 
and bike paths. 


In a two-day visit to Stafford, researchers 
conducted interviews with seven teachers, 
one support staff member, one parent, and 
two school administrators. They also held two 
separate focus groups with seven fifth-grade 
students in total. In addition, the researchers 
observed activities in various spaces, including 
two classrooms during class time, the cafeteria 
during breakfast and lunch, the recess 
playground during breakfast and lunch and the 
front lobby and K-2 corridors during the school 
day. 


This research revealed Stafford to be a school 
without serious behavior problems. For instance, 


the School Progress Report from 2014-15 reveals 
that fewer than 5% of the school’s students 
received an OSS the previous year. The 
counselor confirmed: 
It’s really a pretty calm environment 
around here. If we have a fist fight or 
something, it’s few and far between; 
it’s not like it’s happening daily. And it’s 
usually over something silly, like boys were 
trying to get the ball from each other and 
ended up in a fist fight or something. 


The school’s positive ethos benefits from 

its dedicated staff and parent community. 
Several staff members live in or have lived 

in the surrounding neighborhood, and the 
principal grew up nearby. The school has 

had the fortune of being situated in a strong 
community with invested parents. Teachers 
reported feeling lucky to work there. The staff 
at the school is generally stable and described 
the students as mostly well-behaved, noting the 
strong parental involvement at the school. 


Allin all, however, the data we collected at 
Stafford paints a complex picture: The school 
boasts a low suspension rate—less than half 
the district average—that Is likely a product of 
both a relatively compliant student body and 
the principal’s belief that behavior should be 
managed in the classroom. However, teachers’ 
survey responses suggest that teachers 
believe that suspensions are necessary 

and help maintain order in the building. 

Like teachers at Downing, those at Stafford 
desire a stronger presence from their school 
administration in handling disciplinary issues. 
While teachers expressed an appreciation 

for having the freedom to manage their own 
classroom and students, they also believed 
that the administration should use more severe 
consequences when dealing with serious and 
ongoing student misbehavior. 


Managing behavior in the classroom 


Teachers at Stafford described their principal, 
Ms. Mazzoni, as encouraging and upbeat. 
However, those we spoke with were unanimous 
in the view that her leadership style and 
expectations for discipline can be problematic. 
Whereas the previous principal emphasized 
school-wide expectations, especially in 

the hallways and other common spaces 
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in the building, teachers reported that the 
current principal has placed a much stronger 
emphasis on their managing their own students 
and classrooms. While several mentioned 
feeling thankful to be trusted to handle 

their own classrooms, most simultaneously 
expressed a desire for more support from their 
administration on discipline. One teacher 
expressed concern that students have different 
understandings about the expectations for 
appropriate behavior. She has noticed, she 
reported, an increase in hallway misbehavior. 


To contend with the absence of school-wide 
expectations, some grade-level teams at 
Stafford try to be consistent in addressing 
misbehavior within the grade. These teachers 
work together to enforce consequences for 
students, and some take turns holding teacher- 
administered lunch detentions. Other grade- 
level teams have less coherence. 


The principal reported working to support 
classroom-level behavior management by 
emphasizing proactive support of positive 
behavior at the school level. Allin all, however, 
our case study findings from Stafford reflect 
the survey finding that proactive supports for 
positive behavior at Stafford are relatively 
sparse and non-routinized. 


When behaviors do escalate, Stafford’s 
counselor is often the go-to. The counselor 
herself is somewhat conflicted about her dual 
role as both trusted advisor for students and 
disciplinarian. She reported spending a great 
deal of time talking with students, helping 
them reflect on their actions, and running peer 
mediation and groups for students having 
behavior problems. When problems persist, the 
counselor says, she reaches out to parents. 
She may eventually schedule a meeting that 
includes Mazzoni and may result in a referral for 
supportive services. 

Stafford’s counselor and principal both 
describe the school’s overall disciplinary 
approach as being “progressive,” with 
consequences moving from recess and lunch 
detention to reflection to parent meetings. 
Mazzoni gets involved in individual cases at the 
counselor's request; they always confer before 
suspending a student. 


Suspensions are uncommon at Stafford, 


reserved mainly for serious offenses like 
repeated fighting. Pink slips are only used 

in extreme cases. Typically, referrals to the 
counselor are communicated via email and 
anecdotal notes from teachers. One fifth grade 
teacher explained: 


[Discipline] is pretty much at the teacher 
level. | know that [suspending students] 
goes against our SPR report and other 
school performance profiles, so... we 

try to keep it off the record as much as 
possible when it comes to paperwork and 
stuff like that. 


Regarding the Code of Conduct, the 
counselor said she only referred to it in the 
case of a very serious incident—a weapons 
infraction, for example—to make sure that 
protocol was being followed. For less serious 
incidents, the Code is not referenced. And 
while the principal reported “zero tolerance” 
for physical altercations, she also said that the 
consequence for fighting depends on each 
student's role in the incident. Minor physical 
incidents, like mutual pushing and shoving on 
the playground, would almost never result in a 
suspension. The counselor explained: 


Sometimes when they fight they don't 
get [suspended]. Most of the time we 
deal with it in house here. We find a way 
to deal with it without doing a suspension 
because for most kids a suspension is not 
really going to change their behavior. 
They need fo kind of learn from their own 
... behavior and hopefully make changes 
that way. 


A wish for more support 


Stafford’s teachers, for their part, reported 

that consequences are not administered 
consistently at the school and that serious 
incidents are often not addressed stringently 
enough. While the principal and counselor may 
understand why two students involved in a fight 
do not receive the same consequence, their 
reasoning is less clear to teachers. One third- 
grade teacher expressed the belief that much 
misbehavior goes unaddressed: 


[The principal’s] response is always [that 
we cannot address all misbehavior 
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because of] our lack of staff. There are 
behaviors that need to be dealt with that 
are just kind of pushed to the side. 


Several teachers expressed concern about the 
cumulative impact of letting small incidents 

go unaddressed. Another third-grade teacher 
shared: 


Today | had a report from one of the 
women in the cafeteria that she was 
disciplining a student for running in the 
hallway, and another one of the students 
in my classroom happened to walk by 
and say “look at that ugly woman yelling 
at that girl.” That should probably be 
addressed somewhere along the line but 
that just kind of gets [left] out. Things like 
that are going to escalate. Kids realize 
they get away with these things and they 
just take it fo the next level, and that level 
of respect for adults and the people in the 
building kind-of just goes down. 


Despite some differences in terms of how 
discipline should be handled, teachers and 
administrators at Stafford are united in their 
belief in the school as a positive and strong 
environment for their students. In addition, they 
share a general perception that the support 
they receive is inadequate for addressing the 
disciplinary challenges they face: Teachers wish 
for more support and consistency from their 
principal, who, in turn, notes the need for more 
staff and resources from the District. 
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Case study 4: PBIS in a challenging 
context 


Transition times at Clybourne School can feel 
chaotic, especially on the third floor, which 
houses the middle school in this K-8 building. 
That hallway is loud; students push and 

shove, and teachers call from their classroom 
doorways to move along and get to class. 
These instructions are largely ignored. In mid- 
afternoon, as one travels downstairs from the 
third floor, food wrappers and overturned lunch 
trays litter the stairwell. The lower levels of the 
building have a noticeably calmer climate, but 
raised voices—teachers’ and students’ —echo 
throughout the building. 


Clybourne is situated in a neighborhood that 
faces considerable challenges, including 

high rates of poverty, drug use, and violence. 

It faces serious academic challenges, with 

very small percentages of students scoring 
proficient or above on state assessments. 

All of the school’s students are classified as 
economically disadvantaged, and a larger- 
than-average percentage receive special 
education services. Nearly all Clybourne 
students are African American. A few years 
ago, Clybourne incorporated students from 
another nearby school that was shuttered, 
doubling the school’s population. The influx 

of students from the nearby school motivated 
Clybourne’s long-time principal, Mr. Jackson, to 
pursue PBIS training for his staff. While student 
misbehavior is an ongoing and serious problem, 
the staff believes that behavior and climate 
would be worse without PBIS. 


Our research team visited the school on 

three separate school days in the spring 

of 2017. We observed a meeting of the 
school’s PBIS team, several classrooms, the 
lunchrooms, hallways during transition times, 
and the accommodation room. We spoke in 
interviews and focus groups with four classroom 
teachers, the principal, the special education 
coordinator, the PBIS lead, and two support 
staff. 


Teachers and staff members we spoke with 
emphasized that Clybourne’s student body 
is challenging to work with, often noting that 
students with severe social and emotional 


issues are not receiving services they need. The 
staff also mentioned that some of the behavior 
challenges in the school are likely attributable 
to students being behind academically and 
struggling to complete their work. Staff and 
teachers alike describe their dedication to 
making the school the best place it can be. At 
the same time, many characterized student- 
staff relationships as generally negative. In our 
time in the building, we observed frequent 
looks of exasperation on the part of exhausted 
teachers. 


Enacting the district message 


Clybourne’s principal and teachers 
emphasized that many students’ school 
behavior is influenced by the lack of structure 
they experience at home. An aide said: 


Some of [our students] are out here on their 
own. They're running around. | don’t know 
where the parents could be. | don’t [know] 
what's going on, but a lot of them are going 
through some things, you can tell. And they 
bring it into the school. 


Principal Jackson, who makes alll final decisions 
regarding suspension, uses exclusionary 
disciplinary actions sparingly and only after 
considering the home situations of the students 
involved. He lamented that assigning an OSS 
does not prove to be a useful consequence for 
many students, noting that suspended students 
are often seen riding their bikes up and down 
the block all day. The principal also described 
the risks students may face from being sent 
home: 


!am not putting a kid at home in that 
situation because at the end of the day, 

| come here for kids, | don’t come here 
for adults. | haven't been doing this for 

all these years for grownups. | did this 

to try to change the lives of children so 
why would | say, “you are going to go fo 
your house and be suspended in a home 
where | don’t even think that you're being 
taken care of”? | am not going to do it. 


The school uses the District’s Code of Conduct 
only as a guide, Jackson reported, adding 
that, in some cases, the Code’s recommended 
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consequences for particular behaviors are 
unrealistic given the prevalence of those 
infractions at the school. Ultimately, he does 
what he feels is best in response to misbehavior 
given the particular student and situation. 


Approach to discipline: Working hard to 
make PBIS work 


By most accounts, Clybourne is working to 
implement its PBIS program with fidelity. The 
PBIS team, which includes the PBIS lead, the 
principal, and a number of teachers, meets 
on a monthly basis. The meetings follow 

a structured format: reviewing PBIS data, 
identifying a problem, and then coming up 
with an intervention. The teachers and other 
school staff we spoke with were all aware 

of the school’s token economy system, and 
school-wide expectations for behavior are 
displayed prominently throughout the school 
with specific expectations for different settings. 
Consequences for various kinds of infractions 
have been identified and clarified among 
teachers. 


At Tier Il of PBIS, Clybourne uses “check-in, 
check-out,” an evidence-based system that 
pairs students with behavioral challenges with 
adult mentors, who provide consistent support 
and accountability (Cheney et al., 2009; 
Hawken, Adolphson, Macleod, & Schumann, 
2009; Hawken, MacLeod, & Rawlings, 2007). The 
PBIS lead facilitates a weekly meeting to discuss 
the progress of these students and make 
determinations about next steps. Students who 
show progress in regulating their own behavior 
may “graduate” out of Tier Il; those who fail to 
progress may be referred for more intensive 
behavioral health services. During our visits, 
nearly 20% of the Clybourne student body had 
Tier Il referrals. 


The staff members and teachers we spoke 

with at Clybourne consistently described the 
principal as involved in climate efforts and 
committed to the PBIS program. Some reported 
that the adoption of PBIS has helped shift the 
emphasis at Clybourne to positive rather than 
negative behaviors. The school’s PBIS Lead 
believes it was important to the school’s ability 
to handle the influx of students following the 


nearby school’s closure; it enabled staff, she 
reported, to collectively say, “This is how we do 
things.” 


Despite these successes, members of the 
leadership team reported that the PBIS 
program is only effective for some students 
and that not all teachers adhere to the 
school’s expectations for reinforcing positive 
behavior and using the token economy system 
appropriately. The prevailing belief is that 
behavior and climate in the school would be 
much worse without PBIS, but that the PBIS 
program has not reached its potential due to 
a lack of buy-in and consistency among some 
staff. 


Needed resources: People and space 


Unlike many schools in the District, Clybourne 
has an accommodation room where first- 
through eighth-grade students can be 

sent during the day for 45 minutes or for an 
entire day, depending on the severity of the 
infraction. While teachers are also expected 
to send classwork with their students to the 
accommodation room, in theory the purpose 
of the space is for reflection and restorative 
activities. The day we visited Clybourne’s 
accommodation space, our researchers 
observed yelling and a standoff between 

an aide and a student, but no reflection or 
completion of classwork. 


Despite its relatively well-developed PBIS system 
and the presence of an accommodation 
room—a resource many schools in the SDP 
wish for—Clybourne staff strongly expressed 
the need for more resources. Specifically, 
interviewees reported a dire need for more 
and better trained staff, as well as additional 
space, including a larger and better-equipped 
accommodation room. Several respondents 
cited staff shortages as a main reason for the 
disorder in the hallways. When asked what 
resource would benefit the school the most, 
one staff member responded: 


[We need] people in the hallways, in 
the stairs. The stairs are a mess with food 
because the kids come up out of that 
lunch room and just drop everything 

on the steps. Why? Because there's not 
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enough people to stand at the doors and 
tell them, “Don’t bring that food out of 
the lunch room. Eat in the lunch room.” 
Instead of being in class, kids are in the 
staircase or in the halls because there are 
no people in the halls telling them to get 
out and go back fo class. 


Principal Jackson specifically identified an 
effective and attentive school police officer 
as a valuable resource the school lacks. While 
Clybourne—unlike many elementary and 
middle schools in SDP—does currently have a 
school police officer; Jackson describes him as 
insufficiently engaged. 


A challenging population with a dedicated 
staff doing all they can 


The staff members at Clybourne recognize 
that they are dealing with a challenging 
student population whose lives outside of 
school influence behavior in school. The staff 
described themselves as dedicated to working 
with this population of students and doing all 
they can to help the students be successful, 
but the school could benefit from additional 
staff, soace, and access to behavioral and 
mental health services. PBIS plays a significant 
role in this school’s culture, although it remains 
only a piece of the school’s approach to 
discipline. The accommodation room is viewed 
as necessary and valuable, although who 

it actually benefits is a bit Unclear from our 
observations. The Clybourne staff and Principal 
Jackson appear to still be working on finding 

a coherent and effective strategy for better 
addressing the needs of their student body. 
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Conclusions 


Each of the four cases described in this chapter offers 
a glimpse inside a school that is working to align itself 
with the district's climate and suspension-reduction 
goals. Our profile analysis located three of the four 
schools—Gannis, Downing, and Clybourne—within 
Profile 3 (collaborative and relational); Stafford, 
despite being classified in Profile 1 (reactive and 
autonomous), has a below-average suspension rate. 
The consistencies and inconsistencies, and alignments 
and misalignments, that we observe by looking across 
these four instantiations of SDP-aligned climate efforts 
therefore give rise to useful insights about the barriers 
and facilitators for schools as they work to enact the 
district’s climate agenda. 


A key consistency we observed among all four case- 
study schools concerns the receipt of district messages 
about the need to reduce OSS and improve climate. 
Administrators at all four schools report having heard 
these messages—largely communicated via the 
School Progress Reporting process. How school leaders 
interpret them, however, is less consistent, and these 
inconsistencies are borne out in school-level policy. 

At Stafford, Downing, and Clybourne, principals echo 
district leaders in asserting that suspensions are largely 
ineffective and a last resort. Stafford and Downing’s 
principals have all but prohibited OSS in response, and 
with striking results: Both Stafford and Downing reported 
suspension rates of less than half the district average 
for 2015-16. 


Administrators at Gannis, however, interpret the 
district’s message differently. Rather than “don't 
suspend students,” they understand the directive to 
mean “use suspension only when necessary and be 
sure to follow proper procedures when you do.” This 
school—whose suspension rate for 2015-16 was nearly 
double the district average—does not seem to see 

a conflict between the use of OSS and an otherwise 
collaborative and relational approach to discipline. 
On the contrary: Suspension is viewed as a last resort, 
but also as an important tool for maintaining a positive 
and orderly climate overall despite serving a very 
disadvantaged and challenging student body. 


In all cases, the principals of the schools in our case- 
study sample have responded to district calls for 
reductions in OSS by pushing increased responsibility 
for behavior management to classroom teachers. 
Casually sending a troublesome student to the office 
is, by and large, a thing of the past in all four of 

these schools. At Gannis, teachers describe a sense 

of personal accountability for building their own 
behavior-managemenrt skill sets, and outline steps they 
have taken and supports that assist them in doing so. 


Conversely, teachers at both Stafford and Downing 
report feeling unsupported by administrators on 
discipline, and express the belief that they are being 
asked to do too much. For some teachers, the loss of 
suspension as a useful tool for temporarily removing 
problem students from their classrooms and sending 
strong messages to parents seems emblematic of 
this perceived lack of administrative support. It’s a 
change many teachers meet with bitter resignation. 
At Downing, a “shadow” disciplinary system reflects 
teachers’ preferences for more punitive responses to 
student misbehavior, essentially circumventing the 
restorative climate the principal is trying to foster; 

at Stafford, the relatively trouble-free student body 
obscures the lack of a coherent disciplinary vision. 


Teachers at Clybourne, though outwardly supportive of 
the PBIS principles promoted by the principal, struggle 
and often fail to execute them at the classroom 

level. Clybourne’s student body is particularly 
disadvantaged—significantly more so than those at 
Stafford or Downing, for instance. For teachers at this 
school, the combination of rampant emotional and 
behavioral issues, a suspension rate slightly below the 
district average, and downward pressure on classroom 
behavior management presents a seemingly impossible 
conundrum. Though all the schools in our sample 

face resource limitations, particularly in the area of 
staffing for discipline and student management, it 

is in Clybourne’s case where this need seems most 
pronounced, and where the missed opportunity for 
strong PBIS implementation is most striking. Without 
additional staff and training specifically designed 

to build staff's PBIS skills and buy-in, the likelinood 

of Clybourne ever realizing the full potential of PBIS 
appears slim indeed. 


Two of the four schools in our case-study sample— 
Gannis and Clybourne—are part of SDP’s PBIS initiative. 
Yet these two schools’ enactment of the approach 

is strikingly different. Staff members at Clybourne are 
working to implement the systems and processes of 
PBIS at the school and classroom levels, and have 
progressed to the adoption of Tier Il PBIS systems. 
Implementation, however, is poor; the school’s nascent 
PBIS system seems entirely overwhelmed by the sheer 
volume of day-in, day-out emotional and behavioral 
issues at the school, many of them severe. Conversely, 
our researchers observed that Gannis seems to be 

PBIS “in name only.” Though its student-centered, 
positive climate is receptive to PBIS practices, few are 
actually in use. The strength of the school’s climate, we 
observed, derives more from the staff’s consistent vision 
and connectedness to the students and their families 
than from PBIS per se. And, Gannis’ high suspension 
rate suggests that discipline functions, essentially, at 
two different levels: the caring, relational level that 
works well for most students; and the law-and-order 
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level that removes those students who do not respond. 
Thus Gannis—perhaps the strongest school in our case- 
study sample in terms of its coherent embrace of a 
positive school climate—emerges as neither a strong 
implementer of PBIS nor a suspension-reduction success 
story. 


Thus, our case-study research serves to highlight a few 
key questions, crystallized by the specific contexts of 
these schools: How can school leaders at Downing 
and Stafford bring teachers on board with climate 
goals that make their already challenging jobs even 
more difficult? What would it take to achieve effective 
PBIS implementation in a school as challenged as 
Clybourne? And, how can Gannis maintain a climate 
that is, by all accounts, positive and student-centered 
without resorting to the liberal use of OSS with a few? 


We will return to these questions, and cast them in the 
light of all of our qualitative and quantitative findings, 
in the final chapter of this report. 


51 Consortium for Policy Research in Education | RR 2017 - 4 


Discipline in Context: Suspension, Climate, and PBIS in the School District of Philadelphia 


Chapter Five. A Matter of Context 


In Chapter 1 of this report, we detail SDP’s recent 
efforts to facilitate dramatic shifts in schools’ 
disciplinary practices and climate through a 
combination of policy and programmatic changes. 
Chapter 2 describes some results of these initiatives: 
Elementary- and middle-school leaders report 
awareness of the need to reduce OSS, and of the 
potential benefits of climate initiatives like PBIS. Many 
describe efforts to adopt PBIS in whole or in part, and 
with or without financial support from the district. At 
the same time, this chapter reveals stubborn barriers 
to the realization of SDP’s climate and disciplinary 
goals—chiefly in the form of resource limitations and 
philosophical misalignments within schools. 


In Chapter 3, we explore patterns in schools’ 
approaches to climate and discipline, and find three 
distinct profiles of discipline and climate. The extent 

to which individual schools reflect these patterns 

is related to attributes of their student populations 

and features of their neighborhood contexts. In 
addition, we find that schools’ profiles predict their 
students’ academic achievement and disciplinary 
outcomes. The effect sizes on academic outcomes 
we observe in schools with collaborative climates 

and non-punitive approaches to discipline, relative 

to all other schools, range from .18 to .24 standard 
deviations. Effects of this magnitude are regarded in 
the education literature as significant from a policy 
perspective (Hedges & Hedberg, 2007), and are 
comparable to those observed in most rigorous studies 
of instructional interventions (Lipsey et al., 2012). We 
find, in other words, that a collaborative, relational 
and non-punitive approach to climate and discipline 
is associated not only with decreased risk of students 
receiving OSS, but also with academic-achievement 
benefits that rival those produced by most instructional 
interventions. This is powerful, and underscores the 
urgency of helping schools overcome the obstacles to 
strong implementation of climate programming. 


Chapter 4 explores the role of context with regard to 
climate improvement, with a focus on schools’ deeply 
ingrained assets and challenges. Viewed together, 
Chapters 3 and 4 provide a sense of the tremendous 
diversity of SDP schools’ contexts and highlight the 
persistent and alarming association between schools’ 
obstacles and outcomes and the race and poverty 
level of their students. They underscore one of the 
key conclusions of this study: that efforts to shape 
schools’ climates and approaches to discipline can 
only succeed to the extent that they accommodate 
and adapt to the assets and challenges of particular 
contexts. 


Chapter 4 illustrates that contexts vary widely 

even within SDP. We argue, given this, that tailored 
implementation approaches may help groups of similar 
schools adapt, support, and supplement climate 
programming in order to help initiatives like PBIS take 
hold. And we identify themes that are consistent across 
SDP schools. In order to implement climate initiatives 
well, and in order to reduce or eliminate the use of 
exclusionary practices, we argue that all K-5 and K-8 
schools need: 


1. Strong leadership around climate that consistently 
communicates a clear vision and roles for all staff. 


2. Training and support that can help teachers 
understand the harms of exclusion and embrace 
the goals, purposes, and practices of proactive 
climate-improvement efforts. 


3. Support staff who are numerous enough to be 
visible and trained to carry climate efforts into 
non-instructional spaces. 


4. Places of respite for students who need them, 
including both non-punitive retreats within school 
buildings and mental and behavioral health 
options for needs schools simply cannot meet. 


5. Universal trauma training aimed at helping school 
staff identify and address students’ emotional 
needs in a context where behavioral and 
mental-health services are often inadequate or 
altogether unavailable. 


The findings we identified in the course of this study 
offer support for the promise of PBIS and of SDP’s focus 
on reducing OSS. However, the exploratory nature of 
the research demands caution in how these results are 
interpreted and used. This study was not designed to 
produce confirmatory evidence as to the existence or 
nature of disciplinary and climate profiles within SDP, 
nor does it support causal claims—it would be wrong to 
conclude, for instance, that the climate characteristics 
that define our school profiles cause the differences 
we observed in student outcomes. (We plan future 
experimental and rigorous quasi-experimental research 
in collaboration with SDP that will help us better 
understand causality.) 


Similarly, school and district leaders in SDP should 

resist the temptation to use the profiles we posit here 
to inform conclusions or decisions about particular 
schools. Our findings regarding individual schools’ 
profile assignments are based on surveys with response 
rates of between 50 and 60%, with much higher 
participation from some schools than others. For at 
least some of the schools in our analysis, the response 
rate was well below what would be needed to make 
accurate profile classifications. For this reason, we 
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emphasize the profiles themselves rather than which 
schools belong to each, and we have worked to 
disguise the identities of the individual schools in our 
research. Instead, school leaders would do well to use 
the findings of the report in a more general way, to 
assess their institutions’ strengths and areas for growth, 
and to set aspirational targets. District leaders might 
consider the characteristics of the three profiles in 
establishing benchmarks for schools overall, and for 
thinking generally about the kinds of supports that can 
help schools of each type to move toward successful 
climate improvements. 
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Appendix A: Statistical Analyses 


Latent Class Analysis 


The statistical method used to identify the school 
profiles in response to research question 3 was latent- 
class analysis (LCA). LCA is an analytic method that 
uses a large array of attribute data to identify patterns 
that reflect underlying classes, or profiles (Dayton, 
1998). We applied this method to survey responses 
aggregated at the school level, to identify patterns of 
disciplinary approaches and climate that exist within 
the district. 


Latent class analysis is multivariate statistical method 
used to discover groupings in categorical data. 

LCA involves a process of comparing different 
possible solutions in order to identify the number 

of latent classes—in this case profiles—that best 
captures meaningful variation in the data. The LCA 
model estimates class membership probabilities 

and item-response probabilities conditional on class 
membership. Because LCA is ad measurement model, 
the measurement error is estimated from the vector of 
latent-class membership probabilities and removed. 
We used SAS v9.3 with PROC LCA Version 1.3.2 (Lanza, 
Dziak, Huang, Wagner & Collins, 2015) in which 
parameters are estimated by maximum likelinood. 


To support the interpretability of the profiles, survey 
items were transformed to z-scores and then averaged 
separately by profile. To identify the number of classes 
which best described the latent structure, we examined 
competing solutions with varying numbers of classes. 
To assess the relative fit of of these various solutions, we 
examined several information criteria based on the log 
pseudo-likelinood. The model with three classes was 
preferred by the likelinood-ratio G2 deviance statistic 
and Schwarz Bayesian Information Criterion (BIC; 
Schwarz, 1978). 


The next step in assessing the results of the LCA analysis 
was to explore the classificatory accuracy of the 
best-fit model; that is, the level of success in assigning 
schools to the three profiles. We observed that schools 
in each of the three identified profiles had 0.99 average 
probability of class membership, indicating high 
confidence that profiles were representative of their 
constituent schools. There was no significant difference 
in average probability of class membership between 
the three groups (F (2,163) =0.35, p=0.7081). Moreover, 
we find the average probability for schools to be a 
member of a non-assigned class was less than 0.01. 


Finally, model identification was assessed by examining 
the log-likelinood for many replications with different 
starting values. This process involved estimating the 


same model hundreds of times with different starting 
values for maximum likelinood estimation using the EM 
algorithm. The hope is that most of the replications will 
result in the same small log likelinood statistic, which is 
a measure of the amount of support provided by the 
data for all possible values in our results. Although the 
log-likelinood for the model we report on was among 
the smallest (lowest decile), we note that replicates 
produced a wide range of log-likelihoods. This 
indicates that although the model produces a solution 
with good fit relative to competing specifications, and 
although the model produced excellent classificatory 
accuracy, it is not well identified. As such it is plausible 
that other profile solutions might also be found that fit 
the data nearly as well. 


A key aim of our study is to test the suitability and 
utility of this method for researching school climate 
and discipline. We do not view the resultant profiles as 
confirmatory evidence of the typology, but rather as 
one way of organizing complex data into meaningful 
profiles within the broader mixed methods study. We 
are interested in explicating the profiles based on the 
LCA of survey data in combination with case studies 
to better understand school discipline in context as it 
relates to student disciplinary and academic outcomes. 
To aid interpretation of the profiles, survey items were 
transformed to z-scores and then averaged separately 
by profile. The direction and magnitude of survey item 
means guided the explication and naming of profiles. 


Event History Analysis 


Our preliminary analysis examined the likelinood of a 
student receiving at least one OSS at a given school. 
We estimate survivor functions based on the partial 
likelinood method of fitting a proportional hazards 
model, which allows for heterogeneity in estimating the 
survivor function (Allison, 1995). To increase precision 

of estimates for profile main effects, the model 
included student-level covariates for grade, number 

of unexcused absence (in weeks), number of weeks 
between start of school and start of student enrollment, 
and a set of binary status indicators for gender, Free/ 
Reduced Lunch, Limited English Proficiency, Hispanic, 
African-American, and whether or not the student 
received a prior OSS suspension in the district. 


We can test if the probability that a student is 
suspended is the same in the three school profiles by 
estimating hazard ratios. A hazard is a statistical term 
that refers to the likelihood of an event occurring in 
a given period of time. A hazard ratio is the percent 
change in the hazard comparing once group to 
another. Like an odds ratio, hazard ratios are positive 
and the further the hazard ratio is from 1, the bigger 
the effect it represents. A large positive hazard ratio 
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indicates a high probability of the event occurring 

(i.e. high probability of a student receiving an OSS). A 
hazard ratio less than 1 means that hazard is reduced. 
A decrease in the hazard corresponds to an increase 

in expected survival time, or the period of time during 
which the event does not occur. We conducted the 
statistical analyses in SAS v9.4 using the PROC PHREG 
procedure, which performs regression analysis based on 
the Cox proportional hazard model. 
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